logo
Ask your Symfony questions! Pay money and get answers fast! (more info)

Warning: Please do not give out any FTP or ssh credentials to anyone, unless you trust them completely. Giving out login details is dangerous.

If the asker does not get an answer then they have 10 days to request a refund.

$10
How to get more than 1,000 rows back from Sphinx?

Using Symfony 1.4, and the sfSphinx plugin for search.

I am up against the Sphinx 1,000 row limit (explained here).

I need to be able to get 50,000 rows back.

I opened:

/etc/sphinx/sphinx.conf

and I changed this line:

max_matches = 50000

to 50,000, as you can see.

My PHP code is, sadly, still only returning 1000 rows. What else do I need to change?

I see there is a setLimits() method as part of the sfSphinx plugin:

http://trac.symfony-project.org/browser/plugins/sfSphinxPlugin/trunk/lib/sfSphinxClient.class.php?rev=28182

But I tried calling it and setting the limit to 50,000, but I still only get 1,000 records back, on a query where I should be getting back 8,000 records. You can see my use of setLimits near the bottom of this method:

    private function sphinxSearchForExport($query, $valuesToFilter) {
if($this->sphinxIndexName=='') return array();
//$this->query = ($this->query!='')?implode('* ', explode(' ', $this->query.'*')):'';
$options = array(
'limit' => 50000,
'offset' => ($this->page - 1) * $this->pageSize,
'weights' => array(100, 1),
'sort' => sfSphinxClient::SPH_SORT_EXTENDED,
'sortby' => '@weight DESC',
);
$this->sphinx = new sfSphinxClient($options);

if(!empty($this->filters))
{
foreach ($this->filters as $filter=>$properties)
{
$comparator='=';

if(is_array($properties)) //Check for a multi array.
{
$comparator=(isset($properties['comparator'])?$properties['comparator']:'='); //Comparator, default =
}
else
{
$filter=$properties; //$this->filters is a list (a,b,c)
}

if(!empty($valuesToFilter[$filter]))
{
if ($comparator == 'bet') //Range Comparison
{
$min=(empty($valuesToFilter[$filter]['from_s'])?0:$valuesToFilter[$filter]['from_s']);
//Use max integer 2^32
$max=(empty($valuesToFilter[$filter]['to_s'])?4294967296:$valuesToFilter[$filter]['to_s']);

$this->sphinx->SetFilterRange($filter, $min, $max);
}
else{
$this->sphinx->SetFilter($filter, array($valuesToFilter[$filter]));
}
}
}
}

$this->sphinx->setLimits(0, 50000);
$res = $this->sphinx->Query($this->query, $this->sphinxIndexName);
$this->arrayOfMatches = array();
$this->arrayOfMatches = $res['matches'];

$this->pager = new sfSphinxDoctrinePager($this->class, $options['limit'], $this->sphinx);
$this->pager->setMaxRecordLimit($this->maxRecordLimit);
$this->pager->setPage($this->page);
$this->pager->init();
return (!empty($this->pager)?$this->pager:false);
}



It's in $this->arrayOfMatches where I expect to see more than 1,000 rows, but I'm not getting it.

Lawrence Krubner | 02/28/11 at 4:33pm Edit
Tutorial: How to assign prize money

Previous versions of this question: 02/28/11 at 5:04pm | 02/28/11 at 10:07pm | 02/28/11 at 11:41pm | 03/01/11 at 4:25pm | 03/01/11 at 4:39pm | 03/01/11 at 4:47pm | 03/01/11 at 4:49pm | 03/01/11 at 4:50pm

(1) Responses

See a threaded view of answers?

Warning: Please do not give out any FTP or ssh credentials to anyone, unless you trust them completely. Giving out login details is dangerous.

  • avatar
    Last edited:
    03/03/11
    6:10am
    Loban Rahman says:

    From the link you provided:

    "Second, there's also a 3rd argument to SetLimits() API call that also defaults to 1000 and must be raised. (With SphinxSE, use "maxmatches" option.)"

    Then I checked:

    http://sphinxsearch.com/docs/manual-0.9.9.html#api-func-setlimits

    Which shows that there is indeed a 3rd argument (and 4th!). So do this:

    $this->sphinx->setLimits(0, 50000, 50000);

    But before you do that, find out if this method is a symfony wrapper to the actual sphinx api. If it is, then make sure that this wrapper actually accepts and forwards this 3rd parameter to the main sphinx api. If not, you'll have to modify the plugin to include this.

This question has expired.





Current status of this question: Community pot



Please log in to add additional discourse to this page.





Warning: Please do not give out any FTP or ssh credentials to anyone, unless you trust them completely. Giving out login details is dangerous.

If the asker does not get an answer then they have 10 days to request a refund.