Finding the Need
A few weeks ago I found the http://OpenAustrlia.Org website. I got quite excited when I saw that they offered an API for data sharing. I thought of a few kewl ways I could use that data when I’m blogging.. (hehehe)
Note: MY specific OA API examples require:
- oaapi.php
- jquery-1.3.2.min.js
Finding some examples
I just couldn’t find a generic tutorial on handling XML with PHP. (they probably exist.. But I couldnt find em..)
I saw some excellent tutorials on making parsers for RSS. But somehow the examples just didn’t compute for me. I couldnt workout how to remove the added layers that XML has compared to RSS. (being as Im a cut and paste programmer..)
Then I found this… And some of the blanks filled in..
Original Article
<?php
/**
* A basic example of how to parse XML files.
*
* <p>Relies on PHP's {@link http://php.net/ref.xml SAX parser}.</p>
*
* <p>This example grabs stock quotes from NASDAQ's website. You tell
* the program which stocks to check by adding/removing elements of the
* <var>$Symbols</var> array. The script outputs a series
* of fake SQL query strings.</p>
*
* <p>Output is placed into an associative array called <var>$Data.</var></p>
*
* <p>Element names become the array's keys. So:
* <br /><code><TAG>The info</TAG></code>
* <br />Would become:
* <br /><code>$Data['TAG'] = 'The info';</code>
* </p>
*
* <p>Where elements have attributes, the array's key is the name of
* the element followed by ":" and the name of the attribute:
* <br /><code><TAG ID="55">The info</TAG></code>
* <br />becomes...
* <br /><code>$Data['TAG'] = 'The info';
* <br />$Data['TAG:ID'] = 55;</code>
* </p>
*
* <p>Do note, "case folding" is in effect, so lower case tag and
* attribute names are converted to upper case.</p>
*
* <p>Requires PHP version 4.3.0 or later.</p>
*
* @author Daniel Convissor <danielc@AnalysisAndSolutions.com>
* @copyright The Analysis and Solutions Company, 2002-2004
* @version 2004-02-11 11:00:00
* @link http://www.AnalysisAndSolutions.com/code/phpxml.htm
* @link http://php.net/ref.xml
*/
// ? // Which ticker symbols do you want to evaluate?
$Symbols = array('ET', 'EK');
// Set the base URI.
$URI = 'http://lazerzap.com/oa/representatives.xml';
// Make sure there's no other data with these names.
$ParserProbs = array();
$DataProbs = array();
// Array to convert XML entities back to plain text.
$XmlEntities = array(
'&' => '&',
'<' => '<',
'>' => '>',
''' => '\'',
'"' => '"',
);
/**
* Runs each time an XML element starts.
*/
function StartHandler(&$Parser, &$Elem, &$Attr) {
global $Data, $CData, $XmlEntities;
// Start with empty CData array.
$CData = array();
// Put each attribute into the Data array.
foreach ($Attr as $Key => $Value) {
$Data["$Elem:$Key"] = strtr(trim($Value), $XmlEntities);
echo "$Elem:$Key = {$Data["$Elem:$Key"]}\n";
}
}
/**
* Runs each time XML character data is encountered.
*/
function CharacterHandler(&$Parser, &$Line) {
global $CData;
/*
* Place lines into an array because elements
* can contain more than one line of data.
*/
$CData[] = $Line;
}
/**
* Runs each time an XML element ends.
*/
function EndHandler(&$Parser, &$Elem) {
global $Data, $CData, $DataProbs, $Sym, $XmlEntities;
/*
* Mush all of the CData lines into a string
* and put it into the $Data array.
*/
$Data[$Elem] = strtr( trim( implode('', $CData) ), $XmlEntities);
// echo "$Elem = {$Data[$Elem]}\n";
switch ($Elem) {
case 'LAST-SALE-PRICE':
// Make sure the data is clean.
if ( !preg_match('/^\d{1,8}(\.\d{1,2})?$/', $Data[$Elem]) ) {
// Make note of the error.
$DataProbs[] = "$Elem has bad format: {$Data[$Elem]}";
}
break;
case 'TRADE-DATETIME':
/*
* Ensure data is clean, plus save match parts to $Atom,
* which will be used to convert date/time to MySQL format.
*/
if ( !preg_match('/^(\d{4})(\d{2})(\d{2}) (\d{2}:\d{2}:\d{2})$/',
$Data[$Elem], $Atom) ) {
// Make note of the error.
$DataProbs[] = "$Elem has bad format: {$Data[$Elem]}";
}
$Data[$Elem] = "$Atom[1]-$Atom[2]-$Atom[3] $Atom[4]";
break;
case 'EQUITY-QUOTE':
// Final item tag. Do something with the data.
// Make sure the data is clean.
if ( !preg_match('/^\w{1,9}$/', $Data['EQUITY-QUOTE:CUSIP']) ) {
// Make note of the error.
$DataProbs[] = "$Elem has bad format: "
. $Data['EQUITY-QUOTE:CUSIP'];
}
/*
* Double check that all of the needed data was set.
* If it's not, we don't want to run the query,
* so skip the rest of this section.
*/
if ( !isset($Data['LAST-SALE-PRICE']) ) {
$DataProbs[] = "$Sym LAST-SALE-PRICE wasn't set";
}
if ( !isset($Data['TRADE-DATETIME']) ) {
$DataProbs[] = "$Sym TRADE-DATETIME wasn't set";
}
if ( count($DataProbs) ) {
echo "\nData for $Sym had problems:\n";
echo implode("\n", $DataProbs) . "\n\n";
$DataProbs = array();
} else {
// Construct a sample query string.
$Query = 'UPDATE Quotes SET '
. "TradePrice={$Data['LAST-SALE-PRICE']}, "
. "TradeTime='{$Data['TRADE-DATETIME']}' "
. "WHERE CUSIP='{$Data['EQUITY-QUOTE:CUSIP']}'";
/*
* In the real world, you could run the query now. But,
* for the sake of this exercise, let's just look at it.
*/
echo "$Query\n";
}
}
}
// Loop through each ticker symbol.
foreach ($Symbols as $Sym) {
/*
* Grab the file and stick it into an array.
* Next, check to see that you actually got the raw info.
* Then, implode the raw info into one long string.
*
* If your data is already in string form, you don't need these steps.
*
* This one step requires PHP to be at version 4.3.0 or later.
*/
$Contents = @file_get_contents("$URI$Sym");
if (!$Contents) {
$ParserProbs[] = "$URI$Sym\n Had problem opening file.";
/*
* Start the while loop over again, this time with the
* next item in the $Symbols array.
*/
continue;
}
// debug // echo "\n\n$URI$Sym\n";
// debug // echo "$Contents";
/*
* Take care of characters that choke the parser.
* While I don't think NASDAQ's data poses these problems,
* it's good to keep them in mind.
*/
// Escape ampersands that aren't part of entities.
$Contents = preg_replace('/&(?!\w{2,6};)/', '&', $Contents);
// Remove all non-visible characters except SP, TAB, LF and CR.
$Contents = preg_replace('/[^\x20-\x7E\x09\x0A\x0D]/', "\n", $Contents);
/*
* Clean out the Data array so it can be reused
* to hold the data we parse from the file.
*/
$Data = array();
// Initialize the parser.
$Parser = xml_parser_create('ISO-8859-1');
xml_set_element_handler($Parser, 'StartHandler', 'EndHandler');
xml_set_character_data_handler($Parser, 'CharacterHandler');
// Pass the content string to the parser.
if ( !xml_parse($Parser, $Contents, TRUE) ) {
$ParserProbs[] = "Had problem parsing data for $Sym:\n "
. xml_error_string(xml_get_error_code($Parser));
}
}
// Problems?
if ( count($ParserProbs) ) {
echo "\n" . implode("\n", $ParserProbs);
}
?>
Code Creation?
So with some more examples in my head.. (and my clipboard..)
I created this..
Note: BOTH Left and Right checkmarks must be selected before clicking SUBMIT.
You MUST Click Submit to Send a Request
HTML Form
Here is the (cough) code I used.. Input Form with a Javascript wrapper (I picked up in some somewhere… sorry I forget where.. too many tutorials and code examples in too short a time..)
<html>
<style type="text/css">
#box { background-color:#FFFF99; border:thin solid #FF0000; width:635px; height:50px;}
</style>
<script language="javascript" src="/src/jquery-1.3.2.min.js"></script>
<script language="javascript">
$(document).ready(function() { //Finish loading the entire page before processing any javascript
$("#subBut").click(function(event) {
var formContent = $("#form1").serialize();
$("#box").load('/src/myserv1.php',formContent);
});
});
</script>
</head>
<body>
<table cellpadding="10" border="1"><tr><td>
<h1>Open Australia API Example</h1>
<table cellpadding="8">
<tr><td>
<form name="form1" id="form1" method="post" action="">
<label>
<input type="radio" name="function" value="getDivisions" />
Electoral Division<br />
<input type="radio" name="function" value="getRepresentative" />
MP Details (Person ID # Required)<br />
<input type="radio" name="function" value="getRepresentatives" />
MPs Details<br />
<input type="radio" name="function" value="getSenator" />
Senator Details (Person ID # Required)<br />
<input type="radio" name="function" value="getSenators" />
Senators Details<br />
<input type="radio" name="function" value="getDebates" />
Debates<br />
<input type="radio" name="function" value="getHansard" />
Hansard<br />
<input type="radio" name="function" value="getComments" />
Comments<br /><br>
</td><td>
<label>
<input type="radio" name="search" value="search"> Search<br />
<input type="radio" name="search" value="postcode"> Postcode (4 digits)<br />
<input type="radio" name="search" value="party"> Political Party (Nat,Dem,Gre,Lib,Lab,Ind)<br />
<input type="radio" name="search" value="id"> Person ID # (digits)<br />
<input type="radio" name="search" value="person"> Person (For use with Hansard)<br />
<input type="radio" name="search" value="date"> Date (eg 2009-11-26)<br>
<input type="radio" name="search" value="division"> Electoral Division<br />
<input type="radio" name="search" value="always_return"> Always Return Data<br />
<input type="radio" name="search" value="state"> State</label><br><br />
</td></tr></table>
<label>Search for
<input type="text" name="textfield" id="textfield" />
</label>
<input type="button" name="subBut" id="subBut" value="Submit" />
</form>
</td></tr></table>
<br>
<div id="box">You MUST Click Submit to Send a Request</div>
</body>
</html>
'Dunno What' Parser Code
The above code ‘calls’ my (dunno what) parser. (’/src/myserv1.php’)
<?php
// Include the API binding
require_once 'oaapi.php';
// Set up a new instance of the API binding
$oaapi = new OAAPI('GET YOUR OWN API KEY AND PUT IT HERE');
$function = $_GET['function'];
//$output_type = $_GET['output_type'];
$data = $_GET['textfield'];
$search = $_GET['search'];
// Open Australia API Call
$mps = $oaapi->query($function, array('output' => 'js', $search => $data));
//remove start [
$clstr = substr_replace($mps, "", 0, 1);
//remove end ]
//$clstr2 = substr_replace($clstr, "", -1, 1);
//remove start {
$clstr3 = substr_replace($clstr, "", 0, 1);
//remove end }
//$clstr4 = substr_replace($clstr3, "", -1, 1);
// Output Result
// Get data from #mps
$dataChunks = split("[,]", $clstr3);
for($i = 0; $i < count($dataChunks); $i++){
//echo "$dataChunks[$i] <br />";
$dataChunks1 = split("[:]", $dataChunks[$i]);
for($z = 0; $z < count($dataChunks1); $z++){
//remove start " and store in $cleanedstr
$cleanedstr = substr_replace($dataChunks1[$z], "", 0, 1);
//remove end " from $cleanedstr
$cleanedstr2 = substr_replace($cleanedstr, "", -1, 1);
if ($cleanedstr2 == 'member_id')
{
echo "";
$z++;
}
else
{
if ($cleanedstr2 == '"member_id')
{
echo "<br>";
$z++;
}
else
{
if ($cleanedstr2 == 'person_id')
{
echo "Person ID ";
}
else
{
if ($cleanedstr2 == 'name')
{
echo "";
}
else
{
if ($cleanedstr2 == 'party')
{
echo "Political Party: ";
}
else
{
if ($cleanedstr2 == 'constituency')
{
echo "Electorate: ";
}
else
{
if ($cleanedstr2 == 'house')
{
echo "";
$z++;
}
else
{
if ($cleanedstr2 == 'first_name')
{
echo "";
}
else
{
if ($cleanedstr2 == 'last_name')
{
echo "";
}
else
{
if ($cleanedstr2 == 'full_name')
{
echo "";
$z++;
}
else
{
if ($cleanedstr2 == 'image')
{
echo 'http://openaustralia.org';
}
else
{
if ($cleanedstr2 == 'title')
{
echo "<br>";
}
else
{
if ($cleanedstr2 == 'body')
{
echo "<br><br>";
}
else
{
if ($cleanedstr2 == 'listurl')
{
echo '<br><a href="http://openaustralia.org';
}
else
{
if ($cleanedstr2 == 'speaker')
{
echo '"> Full Hansard Listing</a> ';
}
else
{
if ($cleanedstr2 == 'url')
{
echo '<br><a href="http://openaustralia.org';
}
else
{
if ($cleanedstr2 == 'parent')
{
echo '"> Info on this Speaker</a> ';
}
else
{
if ($cleanedstr2 == 'office')
{
echo '"> Info on this Speaker</a> ';
}
else
{
echo "$cleanedstr2 ";
}}}}}}}}}}}}}}}}}}
//echo "<br>";
}
//echo "<br>";
}
?>
But I was starting to see the limitations of my sloppy code.. I could forsee problems extracting specific data from the stream.
So I decided to train my code skills on some other aspects of data extraction from the API.. (which readers of this blog may notice hanging around political pages in future.. when I get it all sorted..)
Conclusion
Anyway:
- I provide the above code without any claims. It is full of comments that probably dont make much sense.. (But I have other stuff to do.. I may get around to cleaning it one day..)
Note: If a “programmer” would like to refine my example to a few lines of code.. Please do so.. :)
Finding the Need
A few weeks ago I found the http://OpenAustrlia.Org website. I got quite excited when I saw that they offered an API for data sharing. I thought of a few kewl ways I could use that data when I’m blogging.. (hehehe)
Note: MY specific OA API examples require:
Finding some examples
I just couldn’t find a generic tutorial on handling XML with PHP. (they probably exist.. But I couldnt find em..)
I saw some excellent tutorials on making parsers for RSS. But somehow the examples just didn’t compute for me. I couldnt workout how to remove the added layers that XML has compared to RSS. (being as Im a cut and paste programmer..)
Then I found this… And some of the blanks filled in..
Original Article
Code Creation?
So with some more examples in my head.. (and my clipboard..)
I created this..
Note: BOTH Left and Right checkmarks must be selected before clicking SUBMIT.
Open Australia API Example
HTML Form
Here is the (cough) code I used.. Input Form with a Javascript wrapper (I picked up in some somewhere… sorry I forget where.. too many tutorials and code examples in too short a time..)
'Dunno What' Parser Code
The above code ‘calls’ my (dunno what) parser. (’/src/myserv1.php’)
But I was starting to see the limitations of my sloppy code.. I could forsee problems extracting specific data from the stream.
So I decided to train my code skills on some other aspects of data extraction from the API.. (which readers of this blog may notice hanging around political pages in future.. when I get it all sorted..)
Conclusion
Anyway:
Note: If a “programmer” would like to refine my example to a few lines of code.. Please do so.. :)
Add Comment