I'm trying to use curl to automate requesting occupation codes from the Australian Bureau of Statistics web page. I'm getting a "Search Index does not exist" error when I do a POST request with Curl. I think the problem is that there are t开发者_Python百科wo inputs that I'm not entering, and so it is failing. I would appreciate any assistance on this:
Curl request:
curl --data "searchcontent=&helpdocid=[docid]&searchtext=121315&submit=Go" http://www.ausstats.abs.gov.au/ausstats/searchclass.nsf/(Searchattmnt)?openagent
The relevant code from the ABS site is this:
<!-- Start Search Content -->
<div id="titlemain"> Search Results</div><div id="middle"></div>
<form name="attmnt" method="post" action="http://www.ausstats.abs.gov.au/ausstats/searchclass.NSF/(Searchattmnt)?openagent">
<input type="hidden" name="searchcontent" value="">
<input type="hidden" name="helpdocid" value="[docid]">
<div id="content">
<h3>New Search :</h3>
<input type="text" name="searchtext" size =60 MAXLENGTH="255" value="">
<INPUT VALUE="Go" TYPE="submit">
</form>
</div>
<!-- End Search Content -->
Let me know if anything else would help. I'm staring down the barrel of some serious data entry if I can't get this to work.
EDIT: Just fixed up the typo in the curl request to avoid confusion.
The problem is that curl's --data
parameter sends the text exactly as you provide it, but you are not url-encoding reserved characters (specifically, the [
and ]
characters), so the server will not be able to match up the value of the helpdocid
field correctly.
On a side note, don't include the "submit=Go" field in your posted data. Only fields with both names and values get submitted.
Try this:
curl --data "searchcontent=&helpdocid=%5Bdocid%5D&searchtext=121315" http://www.ausstats.abs.gov.au/ausstats/searchclass.nsf/(Searchattmnt)?openagent
I used @Justin Morgan's suggestion and ran wireshark while I performed the request. This gave me the values that I required. In the end this was the curl request I used that worked:
curl \
-d "searchcontent=1220.0Data%2BCubes-SuperTable25.06.091" \
-d "helpdocid=17441354B1295875CA2571E6000AE9A0" \
-d "searchtext=121318" \
http://www.ausstats.abs.gov.au/ausstats/searchclass.NSF/(Searchattmnt)?openagent
The Mozilla plug-in livehttpheader
shows the string of urlencoded text used to post to the server.
精彩评论