Curl cannot post data to php page

I am trying to submit a value to a form to get data from an aspx page, but I cannot get the data

  • First url to open: $url="https://www.clarkcountycourts.us/Anonymous/default.aspx"

    (this is where cookies are set)

  • Second url to find data $url3 ="https://www.clarkcountycourts.us/Anonymous/Search.aspx"

When I post data, I just get the html of page 3 instead of the data from page 2

Below is my code, what am I missing? please guide me.

<?php 
$url="https://www.clarkcountycourts.us/Anonymous/default.aspx"; 
$cookie="cookie.txt";
$url3 ="https://www.clarkcountycourts.us/Anonymous/Search.aspx";

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie);  // <-- add this line
curl_setopt ($ch, CURLOPT_REFERER, $url); 
$result = curl_exec ($ch); 



$SearchBy = "2";
$AttorneySearchMode="Name";
$LastName= "Smith";
$FirstName= "William";
$MiddleName ="";
$CaseStatusType= "0";
$SortBy= "fileddate";
$DateFiledOnAfter = "";
$DateFiledOnBefore = "";
$SearchSubmit ="Search";

$fields = array(
    'SearchBy' => urlencode($SearchBy),
    'AttorneySearchMode' => urlencode($AttorneySearchMode),
    'LastName' => urlencode($LastName),
    'FirstName' => urlencode($FirstName),
    'MiddleName' => urlencode($MiddleName),
    'CaseStatusType' => urlencode($CaseStatusType),
    'SortBy' => urlencode($SortBy),
    'DateFiledOnAfter' => urlencode($DateFiledOnAfter),
    'DateFiledOnBefore' => urlencode($DateFiledOnBefore),
    'SearchSubmit' => urlencode($SearchSubmit)
);

$fields_string = "";
foreach ($fields as $key=>$value) {
    $fields_string .= $key.'='.$value.'&';
}
$fields_string = rtrim($fields_string, '&');

curl_setopt($ch, CURLOPT_URL, $url3);
curl_setopt($ch, CURLOPT_HEADER, true );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,CURLOPT_POST, 1);
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string);
$result2 = curl_exec ($ch); 
print_r($result2);

      

+3


source to share


3 answers


It took a while to figure it out - but the code below should repeat the results after searching based on your original request from William Smith as an attorney. Hope this makes sense ...

            define('ROOT','c:/wwwroot');
            $url_base='https://www.clarkcountycourts.us';
            $url_login=$url_base.'/Anonymous/Login.aspx?ReturnUrl=/Anonymous/default.aspx';
            $url_start=$url_base.'/Anonymous/default.aspx';
            $url_search=$url_base.'/Anonymous/Search.aspx?ID=200&NodeID=101,103,104,105,500,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,797,798&NodeDesc=All%20Courts';
            $cookiejar=tempnam( sys_get_temp_dir(), 'cookiejar_' );

            $lastname='smith';
            $firstname='william';

            $formparams=array(
                '__VIEWSTATEGENERATOR',
                '__EVENTVALIDATION',
                '__EVENTTARGET',
                '__EVENTARGUMENT',
                '__VIEWSTATE'
            );
            $stdparams=array(
                'SearchBy' => '2',
                'AttorneySearchMode' => 'Name',
                'LastName' => $lastname,
                'FirstName' => $firstname,
                'MiddleName' => '',
                'CaseStatusType' => '0',
                'SortBy' => 'fileddate',
                'DateFiledOnAfter' => '',
                'DateFiledOnBefore' => '',                  
                'SearchSubmit' => 'Search',
                'CaseSearchMode' => 'CaseNumber',
                'CaseSearchValue' => '',
                'CitationSearchValue' => '',
                'CourtCaseSearchValue' => '',
                'PartySearchMode' => 'Name',
                'cboState' => 'AA',
                'DateOfBirth' => '',
                'DriverLicNum' => '',
                'chkCriminal' => 'on',
                'chkFamily' => 'on',
                'chkCivil' => 'on',
                'chkProbate' => 'on',
                'chkDtRangeCriminal' => 'on',
                'chkDtRangeFamily' => 'on',
                'chkDtRangeCivil' => 'on',
                'chkDtRangeProbate' => 'on',
                'chkCriminalMagist' => 'on',
                'chkFamilyMagist' => 'on',
                'chkCivilMagist' => 'on',
                'chkProbateMagist' => 'on',
                'DateSettingOnAfter' => '',
                'DateSettingOnBefore' => '',
                'SearchType' => 'PARTY',
                'SearchMode' => 'NAME',
                'NameTypeKy' => 'ALIAS',
                'BaseConnKy' => 'AT',
                'StatusType' => 'true',
                'ShowInactive' => '',
                'AllStatusTypes' => 'true',
                'CaseCategories' => '',
                'RequireFirstName' => 'False',
                'CaseTypeIDs' => '',
                'HearingTypeIDs' => ''
            );
            /* A fudge I know, manually build this param - I think it might be constructed using javascript on submission */
            $stdparams['SearchParams']='SearchBy~~Search+By:~~Attorney~~Attorney||AttorneyNameOption~~Party+Search+Mode:~~Name~~Name||LastName~~Last+Name:~~'.$lastname.'~~'.$lastname.'||FirstName~~First+Name:~~'.$firstname.'~~'.$firstname.'||AllOption~~Case+Status:~~0~~All||selectSortBy~~Sort+By:~~Filed+Date~~Filed+Date ';




            /* Initialise curl and set basic options */
            $curl=curl_init();
            if( parse_url( $url_base,PHP_URL_SCHEME )=='https' ){
                curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 2 );
                curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, false );
                curl_setopt( $curl, CURLOPT_CAINFO, realpath( ROOT . '/cacert.pem' ) );
            }
            curl_setopt( $curl, CURLINFO_HEADER_OUT, true );
            curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
            curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true );
            curl_setopt( $curl, CURLOPT_AUTOREFERER, true );
            curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0' );
            curl_setopt( $curl, CURLOPT_HTTPHEADER, array('Host: '.parse_url( $url_login,PHP_URL_HOST ),'Connection: keep-alive') );


            /* Stage 1 - Get initial page to obtain session cookies etc and store in cookiejar...mmmmm, cookies! */
            curl_setopt( $curl, CURLOPT_URL, $url_login );
            curl_setopt( $curl, CURLOPT_COOKIEJAR, $cookiejar );
            curl_setopt( $curl, CURLOPT_COOKIESESSION, true );
            $res=curl_exec( $curl );



            /* Step 2 - GET form page and analyse input elements */
            curl_setopt( $curl, CURLOPT_URL, $url_search );
            curl_setopt( $curl, CURLOPT_AUTOREFERER, true );
            curl_setopt( $curl, CURLOPT_COOKIEFILE, $cookiejar );
            curl_setopt( $curl, CURLOPT_COOKIESESSION, false );
            $res=curl_exec( $curl );



            /* utility class that simplifies getting DOMDocument with error checking etc */
            $dom=new htmldom( $res );
            $html=$dom->gethtml();
            $col=$html->getElementsByTagName('input');

            /* we need to know values for specific fields - add to array of params to be submitted */
            foreach( $col as $index => $node ){
                if( in_array( $node->getAttribute('name'), $formparams ) ) {
                    $stdparams[ $node->getAttribute('name') ]=$node->getAttribute('value');
                }
            }
            /* Not sure how the 'SearchParams' field value is calculated so calculated manually above..... */




            /* Stage 3 - POST */
            /* Prepare search query */
            $info=$res='';
            $querystring=http_build_query( $stdparams, '', '&' );
            curl_setopt( $curl, CURLOPT_URL, $url_search );
            curl_setopt( $curl, CURLOPT_REFERER, $url_search );
            curl_setopt( $curl, CURLOPT_COOKIEFILE, $cookiejar );
            curl_setopt( $curl, CURLOPT_COOKIESESSION, false );
            curl_setopt( $curl, CURLOPT_POST, true );
            curl_setopt( $curl, CURLOPT_POSTFIELDS, $querystring );
            $res=curl_exec( $curl );
            $info=curl_getinfo( $curl );


            /* The final search results - you could manipulate the dom to get specific items if you wished */
            echo '<pre>';
            print_r( $res );
            echo '</pre>';

            /* Close curl */
            curl_close( $curl );

      



For completeness, the above utility class.

class htmldom{
    private $html;
    public function __construct( $data=false, $convert=true ){
        try{
            if( !$data ) return false;
            libxml_use_internal_errors( true );
            $this->html = new DOMDocument('1.0','utf-8');
            $this->html->validateOnParse=false;
            $this->html->standalone=true;
            $this->html->preserveWhiteSpace=true;
            $this->html->strictErrorChecking=false;
            $this->html->substituteEntities=false;
            $this->html->recover=true;
            $this->html->formatOutput=false;

            $this->html->loadHTML( $convert ? mb_convert_encoding( $data, 'utf-8' ) : $data );

            $parse_errs=serialize( libxml_get_last_error() );
            libxml_clear_errors();

        }catch( Exception $e ){
            die( $e->getMessage() );    
        }
    }
    public function gethtml(){
        return $this->html;
    }
}

      

+5


source


You have two things here.

  • You are not using the cookie jar correctly. CURLOPT_COOKIEJAR

    used when you want to store cookies installed from the server. So cURL doesn't work like a programmable browser. If you want to establish a session (cookie) then the first request uses CURLOPT_COOKIEJAR

    and subsequent requests will useCURLOPT_COOKIEFILE

    CURLOPT_COOKIEJAR

    cookies are placed in the jar from the server.
    CURLOPT_COOKIEFILE

    cookies are sent from the cookie back to the server .

    Using CURLOPT_COOKIEJAR

    outside of the first connection to the server will just overwrite the cookie jar every time.

Cookies and cURL have been a constant source of frustration for me, so I can make a big deal on this first question.

  1. When I manually search this server, the forms end up being submitted to URLs like this:

    https://www.clarkcountycourts.us/Anonymous/Search.aspx?ID=200&NodeID=101,103,104,105,500,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,797,798&NodeDesc=All%20Courts

    Also, when I submit the form in my browser, there are many other fields that are submitted as well.



Some of them look important - __VIEWSTATE

, __EVENTVALIDATION

, __VIEWSTATEGENERATOR

.

__EVENTTARGET:
__EVENTARGUMENT:
__VIEWSTATE:/wEPDwULLTE3NDQ0MTE0OTgPZBYCZg9kFgICAQ8WAh4HVmlzaWJsZWgWAgIDDw9kFgIeB29ua2V5dXAFJnRoaXMudmFsdWUgPSB0aGlzLnZhbHVlLnRvTG93ZXJDYXNlKCk7ZGSj5Ki6eeFgaBABR8RCqLZPLI/WZw==
__VIEWSTATEGENERATOR:B40748C4
__EVENTVALIDATION:/wEWAgKmuqaTBQKYxoa5CF+xFBuOB81vr2zAb/xD7VZ3+CQV
SearchBy:2
CaseSearchMode:CaseNumber
CaseSearchValue:
CitationSearchValue:
CourtCaseSearchValue:
PartySearchMode:Name
AttorneySearchMode:Name
LastName:Smith
FirstName:William
cboState:AA
MiddleName:
DateOfBirth:
DriverLicNum:
CaseStatusType:0
DateFiledOnAfter:
DateFiledOnBefore:
chkCriminal:on
chkFamily:on
chkCivil:on
chkProbate:on
chkDtRangeCriminal:on
chkDtRangeFamily:on
chkDtRangeCivil:on
chkDtRangeProbate:on
chkCriminalMagist:on
chkFamilyMagist:on
chkCivilMagist:on
chkProbateMagist:on
DateSettingOnAfter:
DateSettingOnBefore:
SortBy:fileddate
SearchSubmit:Search
SearchType:PARTY
SearchMode:NAME
NameTypeKy:ALIAS
BaseConnKy:AT
StatusType:true
ShowInactive:
AllStatusTypes:true
CaseCategories:
RequireFirstName:False
CaseTypeIDs:
HearingTypeIDs:
SearchParams:SearchBy~~Search By:~~Attorney~~Attorney||AttorneyNameOption~~Party Search Mode:~~Name~~Name||LastName~~Last Name:~~Smith~~Smith||FirstName~~First Name:~~William~~William||AllOption~~Case Status:~~0~~All||selectSortBy~~Sort By:~~Filed Date~~Filed Date

      

Conclusion

I believe you will need to load the search page in order to get the values ​​to be placed in some of these hidden fields (parsing the page) and then submit the search page again as a POST with the hidden fields filled in.

+5


source


Maybe you are redirected after the initial POST?

Try the following:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);

      

This should ignore any redirects.

Additional CURL options that can help you debug:

CURLOPT_MAXREDIRS
CURLOPT_POSTREDIR

      

Link:

http://php.net/manual/en/function.curl-setopt.php

0


source







All Articles