I have a process that I need to implement on my system which, at a certain point, will check a relatively large number of data and, if it does not find it, should save to the MySQL database.
I'm not finding a way to do this without overloading the server. I'm developing in PHP and so far the only way I thought it was to loop through each "line", checking if there is, if it does not exist, I get the ID, and I write to another table. If it already exists, I only retrieve the ID and saved it in another table.
This way you would have to do 1 query + 1 record + 1 read (retrieve the newly saved ID) + 1 record for each record. If we think that it will be common for each operation to do this on average 3000 times, it becomes unfeasible. Also it will be common to also have more than one user doing this same process at the same time.
What would be the most correct way to proceed in this case?
[Additional Information]
It is a product movement system. Each product has a "serial". So I need to check each serial in table "A" and if it does not exist, I make the register, I get the ID and haul in table "B". If the serial is already registered in the "A" table, I just drop the serial from it in table "B".