How to retrieve a large array in PHP quickly?

Navigation
8

In the example below, I have a array in PHP with about 128,000 records that I retrieved from another file, and I use it in some applications to compare indexes as HashMap . > 

array(128521) {
  [0] =>
  string(3) "aba"
  [1] =>
  string(3) "aba"
  [2] =>
  string(4) "abá"
  [3] =>
  string(7) "ababás"
  [4] =>
  // ...
}

Question

How can I keep this array saved so I can quickly recover it from the hard drive without affecting application performance? The focus is to make array available to classes that will use it as quickly as possible.

    
asked by anonymous 11.01.2014 / 07:34

2 answers

3

In addition to NoSQL options and a Memcached layer as per the comments above, a second option would be search engines. This kind of problem has characteristics that I would deal with with indexing. Do you need the entire array or just query certain words according to the search criteria?

If you need the second option I would use a tool like Apache Solr or Elasticsearch to create a dedicated index (eg, index containing a tuple [hash / word]). Well-matched indexes are very fast and already have intelligent internal cache policies, capable of returning frequent queries "instantly" even in an index with millions of entries.

    
11.01.2014 / 16:32
6

In fact it is completely impracticable to do this.

128 thousand indices in an array, is a large amount to be manipulated and / or compared.

The ideal is to keep such information in the database and to look for it based on a minimum parameter limit.

I have maybe two solutions:

Physical Archive

This array is kept as a physical file (which extension you want, in json format)

[EDITED] Idea

Just as an add-on, in your case, I think it's better to create a file for each "letter" of the alphabet.

  • a.json
  • b.json
  • c.json

Unless your comparison, include values between the value of the array, for example: ada

  • ada
  • adá
  • guava fruit
  • giraffe (joking, haha)

With this I believe that the search is faster (in a single file), but of course, increase the number of requests on the server, and to help in this case, leave any information in the inline (minify) / strong>, so that the size is reduced / compressed to the maximum.

Cached page (json / xml)

  

I do not like and / or recommend xml, but feel free

Have a page (route) of your system / site, which contains this array also in json format, however, such a page will be cached by the time you set.

In this way, when other applications access the specific page (for example, http://www.examplo.com.br/ptbr.json or http://www.examplo.com.br/ptbr.php - the extension itself does not matter, but how you will handle the request), you return this page, which in this if it is cached, it will not load again.

Doubt

Do you use any frameworks? The vast majority of them already have a ready cache system, and excellent administration of the extension / response / protocol type of your request / route.

Some links to help you

I hope I have helped.

    
11.01.2014 / 13:32
How to increase or decrease div proportional to 720x540? How to display the exception message (ex.getMessage ()) thrown by the server in a jquery ajax?