Message Boards Message Boards

0
|
6447 Views
|
1 Reply
|
0 Total Likes
View groups...
Share
Share this post:

Using URLfetching, to get Web data (VIZier) is possible?

Posted 10 years ago

Hi,

I have to get parallaxes position of a lot of stars form Vizier Catalog: http://vizier.u-strasbg.fr/viz-bin/VizieR-2;

I wonder if it would be possible to use URLfetch function like that:

URLFetch["http://vizier.u-strasbg.fr/viz-bin/VizieR-2", 
 "Parameters" -> {"Astronomy" -> "Parallaxes", 
   "Target Name" -> "1 cam A"}]

When I used it, it just output the XML page

POSTED BY: Marcelo De Cicco

I decide to use Import, and really works:

First I did a dataName.....dat, with all star names (In the GOSC html format: http://ssg.iaa.es/en/content/gosc-v3-query), that I intended to get the informations I needed: Ra, Declination,Paralax,etc..

Doing the following:

I choose the aladin format, in the GOSC Data Base url. I download it:

And I did some reduction:

dataO = StringSplit[ Import["C:\Users\...........output20140928054517. aladin", "Data"], "|"];

Then, I did the following:

(*FORMATTING CSV E DAT*)
Export["dataO.csv", dataO];
Export["dataO.DAT", dataO];
dataOcsv = Import["dataO.csv"];

TableForm[dataOcsv];

(*TABLE DATA  - CHOOSEN VARIABLES : NOME, TYC,RA, ....
DEG, LONG GALATICA E LATITUDE GALATICA*)
(*Obs.: TYC:Tycho-2 catalog number.The link points to the VizieR \
selection page with Hipparcos-and Tycho-related catalogs.*)

dataO2 = TableForm[dataOcsv[[All, {1, 16, 19, 20, 21, 22, 23, 24}]]];
Export["dataO2.csv", dataO2];
Export["dataO2.dat", dataO2];

dataO2csv = Import["dataO2.csv"];
TableForm[dataO2csv];

dataNomeEstrelas = StringTrim[dataO2csv[[4 ;;, 1]]];
DumpSave["dataNomeEstrelas.dat", dataNomeEstrelas];


data.... = 
 Table[Cases[
   Import[url1 <> ToString[URLEncode[dataNomeEstrelas[[i]]]] <> url2 <>
      ToString[URLEncode[dataNomeEstrelas[[i]]]] <> url3 <> 
     ToString[URLEncode[dataNomeEstrelas[[i]]]] <> url4 <> 
     ToString[URLEncode[dataNomeEstrelas[[i]]]] <> url5 <> 
     ToString[URLEncode[dataNomeEstrelas[[i]]]] <> url6, 
    "FullData"], {_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _?
     StringQ}, \[Infinity]], {i, .....,.....) }]
Export["dataParalax......dat", dataParalax.....];

And the Url is

"http://vizier.u-strasbg.fr/viz-bin/VizieR-4?-ref=VIZ542a083a4a04&-to=4&-from=-3&-this=-3&%2F%2Fkw.Astronomy=Parallaxes&%2F%2Fc=1cam+A&%2F%2Ftables=I%2F311%2Fhip2&%2F%2Ftables=I%2F311%2Fhip7p&%2F%2Ftables=I%2F311%2Fhip9p&%2F%2Ftables=I%2F311%2Fhipvim&-out.max=50&%2F%2FCDSportal=http%3A%2F%2Fcdsportal.u-strasbg.fr%2FStoreVizierData.html&-out.form=HTML+Table&-out.add=r&-out.add=RAJ%2CDEJ&%2F%2Foutaddvalue=default&-sort=r&-oc.form=sexa&-nav=cat%3AI%2F311%26tab%3A{I%2F311%2Fhip2}%26tab%3A{I%2F311%2Fhip7p}%26tab%3A{I%2F311%2Fhip9p}%26tab%3A{I%2F311%2Fhipvim}%26key%3Akw.Astronomy%3DParallaxes%26key%3Ac%3D1cam+A%26pos%3A1cam+A%28+++2+arcmin+J2000%29%26HTTPPRM%3A%26-ref%3DVIZ542a083a4a04%26-out.max%3D50%26-out.form%3DHTML+Table%26-out.add%3Dr%26-out.add%3DRAJ%2CDEJ%26-sort%3Dr%26-order%3DI%26-oc.form%3Dsexa%26-kw.Astronomy%3DParallaxes%26-c%3D1cam+A%26-c.eq%3DJ2000%26-c.r%3D++2%26-c.u%3Darcmin%26-c.geom%3Dr%26-file%3D.%26-meta.ucd%3D2%26-meta%3D1%26-meta.foot%3D1%26-usenav%3D1%26-bmark%3DGET%26&-c=14+Cep&-c.eq=J2000&-c.r=++2&-c.u=arcmin&-c.geom=r&-source=I%2F311%2Fhip2%2CI%2F311%2Fhip7p%2CI%2F311%2Fhip9p%2CI%2F311%2Fhipvim&-order=I&recno=&-out=HIP&HIP=&-out=nHIP&nHIP=&-out=Sn&Sn=&-out=So&So=&Nc=&-out=RArad&RArad=&eRArad=&-out=DErad&DErad=&eDErad=&-out=Plx&Plx=&-out=ePlx&ePlx=&-out=pmRA&pmRA=&epmRA=&-out=pmDE&pmDE=&epmDE=&Ntr=&F2=&F1=&var=&-out=Hpmag&Hpmag=&eHpmag=&sHp=&VA=&-out=B-V&B-V=&eB-V=&V-I=&-out=HIP1&-ignore=HIP1%3D&HIP1=HIP1&-ignore=Phot%3D&Phot=Phot&-out=Fg&Fg=&-out=dpmRA&dpmRA=&-out=dpmDE&dpmDE=&-out=edpmRA&edpmRA=&-out=edpmDE&edpmDE=&-out=ddpmRA&ddpmRA=&-out=ddpmDE&ddpmDE=&-out=eddpmRA&eddpmRA=&-out=eddpmDE&eddpmDE=&-out=upsRA&upsRA=&-out=upsDE&upsDE=&-out=eupsRA&eupsRA=&-out=eupsDE&eupsDE=&%2F%2Fnoneucd1p=on&-file=.&-meta.ucd=2&-meta=1&-meta.foot=1&-usenav=1&-bmark=GET"

But using above , is very trouble, so i did this:

url1 = "http://vizier.u-strasbg.fr/viz-bin/VizieR-4?-ref=\
VIZ542a083a4a04&-to=4&-from=-3&-this=-3&%2F%2Fkw.Astronomy=Parallaxes&\
%2F%2Fc="; url2 = 
"&%2F%2Ftables=I%2F311%2Fhip2&%2F%2Ftables=I%2F311%2Fhip7p&%2F%\
2Ftables=I%2F311%2Fhip9p&%2F%2Ftables=I%2F311%2Fhipvim&-out.max=50&%\
2F%2FCDSportal=http%3A%2F%2Fcdsportal.u-strasbg.fr%2FStoreVizierData.\
html&-out.form=HTML+Table&-out.add=_r&-out.add=_RAJ%2C_DEJ&%2F%\
2Foutaddvalue=default&-sort=_r&-oc.form=sexa&-nav=cat%3AI%2F311%26tab%\
3A{I%2F311%2Fhip2}%26tab%3A{I%2F311%2Fhip7p}%26tab%3A{I%2F311%2Fhip9p}\
%26tab%3A{I%2F311%2Fhipvim}%26key%3Akw.Astronomy%3DParallaxes%26key%\
3Ac%3D";
url3 = "%26pos%3A";
url4 = "%28+++2+arcmin+J2000%29%26HTTPPRM%3A%26-ref%3DVIZ542a083a4a04%\
26-out.max%3D50%26-out.form%3DHTML+Table%26-out.add%3D_r%26-out.add%\
3D_RAJ%2C_DEJ%26-sort%3D_r%26-order%3DI%26-oc.form%3Dsexa%26-kw.\
Astronomy%3DParallaxes%26-c%3D";
url5 = "%26-c.eq%3DJ2000%26-c.r%3D++2%26-c.u%3Darcmin%26-c.geom%3Dr%\
26-file%3D.%26-meta.ucd%3D2%26-meta%3D1%26-meta.foot%3D1%26-usenav%\
3D1%26-bmark%3DGET%26&-c=";
url6 = "&-c.eq=J2000&-c.r=++2&-c.u=arcmin&-c.geom=r&-source=I%2F311%\
2Fhip2%2CI%2F311%2Fhip7p%2CI%2F311%2Fhip9p%2CI%2F311%2Fhipvim&-order=\
I&recno=&-out=HIP&HIP=&-out=n_HIP&n_HIP=&-out=Sn&Sn=&-out=So&So=&Nc=&-\
out=RArad&RArad=&e_RArad=&-out=DErad&DErad=&e_DErad=&-out=Plx&Plx=&-\
out=e_Plx&e_Plx=&-out=pmRA&pmRA=&e_pmRA=&-out=pmDE&pmDE=&e_pmDE=&Ntr=&\
F2=&F1=&var=&-out=Hpmag&Hpmag=&e_Hpmag=&sHp=&VA=&-out=B-V&B-V=&e_B-V=&\
V-I=&-out=HIP1&-ignore=HIP1%3D*&HIP1=HIP1&-ignore=Phot%3D*&Phot=Phot&-\
out=Fg&Fg=&-out=dpmRA&dpmRA=&-out=dpmDE&dpmDE=&-out=e_dpmRA&e_dpmRA=&-\
out=e_dpmDE&e_dpmDE=&-out=ddpmRA&ddpmRA=&-out=ddpmDE&ddpmDE=&-out=e_\
ddpmRA&e_ddpmRA=&-out=e_ddpmDE&e_ddpmDE=&-out=upsRA&upsRA=&-out=upsDE&\
upsDE=&-out=e_upsRA&e_upsRA=&-out=e_upsDE&e_upsDE=&%2F%2Fnoneucd1p=on&\
-file=.&-meta.ucd=2&-meta=1&-meta.foot=1&-usenav=1&-bmark=GET";

ToString[URLEncode[dataNomeEstrelas[[i]]]]

The above code, get the name of star, encode it to url coding and transform it a string to be used in the url Vizier query.

Problems:

1 .I do not know how keep the greek letters to be used in the encoded url form; 2.Some stars to not have data in Vizier Data base, so the code returns ,{}, so troubles my table formatting, I really did not think how to set this.

  1. If I try 400 stars, the code does not finish, I have to get 100 stars each runnig, I do not know if is a connection problem or from mathematica Import.
POSTED BY: Marcelo De Cicco
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract