A "Hello World" in the VO

"Naive" tutorial: detailed and super simplified development of
a VO Simple ConeSearch Service from scratch.

Tutorial for the Second European Data Provider Forum and Training Event. 27 - 28 June 2018, Heidelberg (Germany). By Carlos Rodrigo
Last update: June 30, 2018

(You can download this tutorial and all the related files as a .tgz file.)

1.- Install apache-mysql-php (LAMP server in Linux)

This is out of the scope of this tutorial.

We asume that apache-mysql-php are already installed.

Many tutorials in the web (google for "install lamp in...")

For instance, for ubuntu:

In general it is as easy as:

  $ sudo apt-get install tasksel
  $ sudo tasksel install lamp-server

or

  $ sudo apt install apache2
  $ sudo apt install mysql-server mysql-client
  $ sudo apt install php-pear php-fpm php-dev php-zip php-curl php-xmlrpc php-gd php-mysql php-mbstring php-xml libapache2-mod-php

  $ sudo service apache2 restart 

2.- Create a mysql database and a user authorized to handle it.

For this tutorial we will need:

$ mysqladmin -u root -p create mydb
  Enter password:

$ mysql -u root -p 
  Enter password:

mysql> grant all privileges
             on mydb.mytest
             to 'heiuser'@'localhost'
             identified by 'heipass';
mysql>exit

3.- Create a mysql table

We have a CSV file (mytest.csv) with 9 columns:

  1. Identifier (IAU standard)
  2. Right Ascension (ICRS) (degrees)
  3. Declination (ICRS) (degrees)
  4. CMC15 r magnitude
  5. Standard deviation of r magnitude
  6. Mean epoch of the astrometry
  7. 2MASS J magnitude
  8. 2MASS H magnitude
  9. 2MASS Ks magnitude

We create a file named create_table.my (the name is not important) with the definition of the columns that we need in the database:

drop table if exists mytest;

create table mytest (
  id       int not null auto_increment,
  objID    char(255),
  RAdeg    double,
  DECdeg   double,
  magr     double,
  sig_magr double,
  epoch    int,
  magJ     double,
  magH     double,
  magKs    double,
  primary key (id),
  index idx_ra (RAdeg),
  index idx_dec (DECdeg));

We can use this file to easily create the table in the database

$ mysql -u root -p mydb < create_table.my
  Enter password:

4.- Populate the database with the data in mytest.csv

We can use the mysqlimport command to load all the content of the CSV file into the mysql table.

$ mysqlimport -u root -p --local --fields-terminated-by=, --columns=objID,RAdeg,DECdeg,magr,sig_magr,epoch,magJ,magH,magKs mydb mytest.csv
  Enter password:

help on mysqlimport:

$ mysqlimport --help

or, for instance:

5.- Study the relevant VO documents:

others would be good too, for instance:

6.- Make a php code to implement the service

go to somewhere under the web documents folder (usually /var/www/html/ ), for instance:

$ cd /var/www/html/vocats/helloworld/

open a text editor:

$ emacs cs0.php &

and we write an extremely simple PHP program:

cs0.php:

<?php

echo "Hello!"

?>

If you are reproducing this tutorial in your own machine, for instance, your laptop, all url's will start by "http://localhost/vocats/helloworld/...". That is, the path of the files in your computer replacing "/var/www/html/" (the path to the base document folder of your Apache web server) by "http://localhost/".

Instead, if your machine has a real internet domain asigned (for instance "mydomain.net"), all url's will start as "http://www.mydomain.net/vocats/helloworldy/..." (the same than above but replacing "http://localhost/" by "http://www.mydomain.net/".

Here, all captures will show "http://localhost/" in the url's.

When we open it in a browser we should see a simple "Hello!":

We edit it and introduce the basic structure:

cs1.php:

<?php

error_reporting(E_ALL ^ ( E_NOTICE | E_WARNING | E_DEPRECATED));

$RA=$_REQUEST['RA'];
$DEC=$_REQUEST['DEC'];
$SR=$_REQUEST['SR'];

if(check_params($RA,$DEC,$SR)==0)
  {
    show_error("Error");
  }
else
  {
    $data=make_mysql_query($RA,$DEC,$SR);

    show_votable($data);
  }

function check_params($RA,$DEC,$SR)
{
  return 0;
}

function make_mysql_query($RA,$DEC,$SR)
{
  return array();
}

function show_votable($data)
{
  echo "This is the votable";
}

function show_error($text)
{
  echo "one error: $text";
}

?>

when we open it in a browser we see an error because the function "check_params" returns always 0 (by now)

cs2.php:

<?php

error_reporting(E_ALL ^ ( E_NOTICE | E_WARNING | E_DEPRECATED));
header('Content-type: text/xml');

$RA=$_REQUEST['RA'];
$DEC=$_REQUEST['DEC'];
$SR=$_REQUEST['SR'];

if(check_params($RA,$DEC,$SR)==0)
  {
    show_error("Error in input values. You need to specify RA, DEC, SR with values so that RA:(0,180), DEC:(-90,90), SR:(0,180)");
  }
else
  {
    $data=make_mysql_query($RA,$DEC,$SR);

    show_votable($data);
  }

function check_params($RA,$DEC,$SR)
{
  return 0;
}

function make_mysql_query($RA,$DEC,$SR)
{
  return array();
}

function show_votable($data)
{
  echo "This is the votable";
}

function show_error($text)
{
  echo '<?xml version="1.0"?>'."\n";
  echo '<!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd">'."\n";
  echo '<VOTABLE version="1.0">'."\n";
  echo '<DESCRIPTION>This is a test SCS Service</DESCRIPTION>'."\n";

  echo '<INFO ID="Error" name="Error" value="'.$text.'"/>'."\n";

  echo '</VOTABLE>'."\n";
}

?>

We implement the real check of input parameters and a simple votable when inputs are ok

cs3.php:

<?php

error_reporting(E_ALL ^ ( E_NOTICE | E_WARNING | E_DEPRECATED));
header('Content-type: text/xml');

$RA=$_REQUEST['RA'];
$DEC=$_REQUEST['DEC'];
$SR=$_REQUEST['SR'];

if(check_params($RA,$DEC,$SR)==0)
  {
    show_error("Error in input values. You need to specify RA, DEC, SR with values so that RA:(0,180), DEC:(-90,90), SR:(0,180)");
  }
else
  {
    $data=make_mysql_query($RA,$DEC,$SR);

    show_votable($data);
  }

function check_params($RA,$DEC,$SR)
{
  if(strlen($RA)==0 || strlen($DEC)==0 || strlen($SR)==0 || 
     $RA < 0   || $RA > 180 ||
     $RA < -90 || $DEC > 90 ||
     $SR <= 0  || $SR > 180 )
    {
      return 0;
    }
  else
    {
      return 1;
    }

}

function make_mysql_query($RA,$DEC,$SR)
{
  return array();
}

function show_votable($data)
{
  echo '<?xml version="1.0"?>'."\n";
  echo '<!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd">'."\n";
  echo '<VOTABLE version="1.0">'."\n";
  echo '<DESCRIPTION>This is a test SCS Service</DESCRIPTION>'."\n";
  echo '<INFO ID="ok" name="ok" value="GOOOOOD"/>'."\n";
  echo '</VOTABLE>'."\n";

}

function show_error($text)
{

  echo '<?xml version="1.0"?>'."\n";
  echo '<!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd">'."\n";
  echo '<VOTABLE version="1.0">'."\n";
  echo '<DESCRIPTION>This is a test SCS Service</DESCRIPTION>'."\n";
  echo '<INFO ID="Error" name="Error" value="'.$text.'"/>'."\n";
  echo '</VOTABLE>'."\n";

}

?>

Finally, we implement the real mysql search and, once we have the data, we format them correctly in show_votable()

cs4.php:

<?php

error_reporting(E_ALL ^ ( E_NOTICE | E_WARNING | E_DEPRECATED));
header('Content-type: text/xml');

$RA=$_REQUEST['RA'];
$DEC=$_REQUEST['DEC'];
$SR=$_REQUEST['SR'];

if(check_params($RA,$DEC,$SR)==0)
  {
    show_error("Error in input values. You need to specify RA, DEC, SR with values so that RA:(0,180), DEC:(-90,90), SR:(0,180)");
  }
else
  {
    $data=make_mysql_query($RA,$DEC,$SR);

    show_votable($data);
  }

function check_params($RA,$DEC,$SR)
{
  if(strlen($RA)==0 || strlen($DEC)==0 || strlen($SR)==0 || 
     $RA < 0   || $RA > 180 ||
     $RA < -90 || $DEC > 90 ||
     $SR <= 0  || $SR > 180 )
    {
      return 0;
    }
  else
    {
      return 1;
    }

}

function make_mysql_query($RA,$DEC,$SR)
{

  $host='localhost';
  $user='heiuser';
  $pass='heipass';
  $database='mydb';
  $table='mytest';

  $link = mysqli_connect($host,$user,$pass,$database) or die ("<br>Could not connect to database server");

  $query=sprintf("select * from $table limit 10");

  $RA=sprintf("%g",$RA);
  $DEC=sprintf("%g",$DEC);
  $SR=sprintf("%g",$SR);
   
  $query = "select *, (180/pi())*acos(cos((90-DECdeg)*pi()/180)*cos((90-$DEC)*pi()/180)+
           sin((90-DECdeg)*pi()/180)*sin((90-$DEC)*pi()/180)*cos((RAdeg-$RA)*pi()/180)) as pdis 
           from $table 
           where ((RAdeg>=($RA-$SR) and RAdeg<=($RA+$SR)) or (RAdeg-360>=($RA-$SR) and 
           RAdeg-360<=($RA+$SR))) and DECdeg>=($DEC-$SR) and DECdeg<=($DEC+$SR) 
           having pdis < $SR 
           order by pdis 
           limit 10";

  $results = mysqli_query($link,$query) or die("<br>select query aborted");

  $i=0;
  while($row = mysqli_fetch_assoc($results))
    {
      $data[$i]=$row;
      $i++;
    }

  mysqli_close($link);
    
  return $data;

}

function show_votable($data)
{
  echo '<?xml version="1.0"?>'."\n";
  echo '<!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd">'."\n";
  echo '<VOTABLE version="1.0">'."\n";
   
  echo '  <RESOURCE>'."\n";
  echo '    <DESCRIPTION>'."\n";
  echo '';
  echo '    </DESCRIPTION>'."\n";

  echo '    <TABLE>'."\n";
  echo '      <FIELD name="objID"    datatype="char" arraysize="*"   ucd="ID_MAIN"/>'."\n";
  echo '      <FIELD name="RAdeg"    datatype="double" unit="degree" ucd="POS_EQ_RA_MAIN"/>'."\n";
  echo '      <FIELD name="DECdeg"   datatype="double" unit="degree" ucd="POS_EQ_DEC_MAIN"/>'."\n";
  echo '      <FIELD name="magr"     datatype="double"  ucd="phot.mag"/>'."\n";
  echo '      <FIELD name="sig_magr" datatype="double"  ucd="stat.error;phot.mag"/>'."\n";
  echo '      <FIELD name="magJ"     datatype="double"  ucd="phot.mag;em.IR.J"/>'."\n";
  echo '      <FIELD name="magH"     datatype="double"  ucd="phot.mag;em.IR.H"/>'."\n";
  echo '      <FIELD name="magKs"    datatype="double"  ucd="phot.mag;em.IR.K"/>'."\n";
  echo '      <FIELD name="epoch"    datatype="double"  ucd="time.epoch"/>'."\n";
  echo '      <DATA>'."\n";
  echo '        <TABLEDATA>'."\n";

  for($i=0;$i<count($data);$i++)
    {
      echo '          <TR>'."\n";
      echo '            <TD>'.$data[$i]['objID'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['RAdeg'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['DECdeg'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['magr'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['sig_magr'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['magJ'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['magH'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['magKs'].'</TD>'."\n";
      echo '            <TD>'.$data[$i]['epoch'].'</TD>'."\n";
      echo '          </TR>'."\n";
    }

  echo '        </TABLEDATA>'."\n";
  echo '      </DATA>'."\n";
  echo '    </TABLE>'."\n";

  echo '  </RESOURCE>'."\n";
  echo '</VOTABLE>'."\n";	

}

function show_error($text)
{

  echo '<?xml version="1.0"?>'."\n";
  echo '<!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd">'."\n";
  echo '<VOTABLE version="1.0">'."\n";
  echo '<DESCRIPTION>This is a test SCS Service</DESCRIPTION>'."\n";
  echo '<INFO ID="Error" name="Error" value="'.$text.'"/>'."\n";
  echo '</VOTABLE>'."\n";

}


?>

where we have used the CDS UCD Builder to find adequate UCD's for our columns

and when we run a correct query we obtain the desired VOTable with the answer.

8.- Create a new user/authority in the registry

Choose one of the available registries, for instance: Euro-VO Registry

Register a new user:

9.- Register our ConeSearch service in the registry