PHP can not process heavy files

Question

PHP can not process heavy files

Navigation

#1 by (1 votes)

0

I have a system in PHP that reads a TXT file, and performs a REPLACE in the database for each line, it does it well with small files (about 100 lines), however when I do with giant files (more than 1 Million lines) Chrome will have a white screen and will not work. How do I make something big in the browser?

This is an example line:

2018-09-03;AALR3                                             ;0000000090; 000000000012.690000;000000000000000400;10:01:00.021;1;2018-09-03;000086402559117;000000000200197;2;2018-09-03;000086402555935;000000000200198;2;0;00000003;00000003

I only use the first 7 information, so I use this code below:

<?php

$conexao = mysqli_connect("generic_ip", "genericlogin", "generickey", "genericdatabase");

if ($_POST) {
    $count=0;
    $tmp_name = $_FILES["arquivo"]["tmp_name"];
    $ler = file($tmp_name);
    $total = count($ler);
    $controler = 0;

    foreach ($ler as $linhas) {
if($controler > 0){
        $base = $linhas;
        $corrigibase = (explode(";", $base));
        $dado0 = trim($corrigibase[0]);
        $dado1 = trim($corrigibase[1]);


        $dado2 = trim($corrigibase[2]);
        $dado2 = floatval($dado2);
        $dado3 = trim($corrigibase[3]);
        $dado3 = floatval($dado3);
        $dado4 = trim($corrigibase[4]);
        $dado4 = floatval($dado4);


        $dado5 = trim($corrigibase[5]);
        $dado6 = trim($corrigibase[6]);

        $QUERY_MORE = "REPLACE INTO genericdatabase.generictable (Data, Papel, N_neg, Preco, Quant_Neg, Hora, Ind_anul) VALUES ('$dado0', '$dado1', '$dado2', '$dado3', '$dado4', '$dado5', '$dado6')";

        $INSERT_BASE = mysqli_query($conexao, $QUERY_MORE);

$count++;
        $dado0 = NULL;
        $dado1 = NULL;
        $dado2 = NULL;
        $dado3 = NULL;
        $dado4 = NULL;
        $dado5 = NULL;
        $dado6 = NULL;
}
$controler++;

    }

    echo $count."registros contados";

}

?>

php mysql google-chrome txt

asked by anonymous 28.11.2018 / 13:41

1 answer

Images are not displayed, even with correct path Taking only the "src" attribute

score 1 · Answer 1

Yes, with 1 million lines it is already necessary to do the operations by stream line by line, the main modification would be the following lines:

//abrir com fopen ou invés do file()
$ler = fopen($tmp_name, 'r');

//usar o while com fgets ao invés do foreach do seu código pra percorrer
//cada linha separadamente, não carregando todo o arquivo em memória
while ($linha = fgets($ler)) { ... }

//fecha o arquivo depois do while
fclose($ler);