Error reading text file line by line

Question

Error reading text file line by line

Navigation

#1 by (1 votes)

2

I'm trying to read a very large file line by line and adding to a list, however when it arrives at a certain point I get an error: System.OutOfMemoryException

Is there any way to read this whole file in one go?

while ((line = trPlaces.ReadLine()) != null)
{
    var htmlLine = line.Split('\t');

    try
    {
        var newPlace = new Place
        {
            Id = htmlLine[0],
            Name = htmlLine[2],
            IsoCountry = htmlLine[8],
            Latitude = htmlLine[4],
            Longitude = htmlLine[5],
            Admin1 = htmlLine[10].Length == 1 ? "0" + htmlLine[10] : htmlLine[10],
            Admin2 = htmlLine[11],
            PlaceType = htmlLine[7],
            AlternativeNames = htmlLine[3].Split(',')
        };

        if(listPlaces1.Count< 4000000)
        {
            listPlaces1.Add(newPlace);
        } else
        {
            listPlaces2.Add(newPlace);
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex);
    }
}

c# .net

asked by anonymous 30.09.2017 / 20:25

1 answer

Adaptive Storyboard (WPF) How to make my code read in UTF-8 in C?

score 1 · Answer 1

An option to read the whole file at once and load it into an array is:

var arrLinhas = File.ReadAllLines(@"C:\Desenv\Texto.txt");

One suggestion to save memory would be to not load all objects in the list at once. You could create a class to instantiate objects only when they are used.

For example:

public class PlaceFactory
{
    private string[] _places;

    public PlaceFactory(string filepath)
    {
        _places = File.ReadAllLines(filepath);  
    }

    public Place ObterPlace(int index)
    {       
        var htmlLine = _places[index].Split('\t');

        var place = new Place()
        {
            Id = htmlLine[0],
            Name = htmlLine[2],
            IsoCountry = htmlLine[8],
            Latitude = htmlLine[4],
            Longitude = htmlLine[5],
            Admin1 = htmlLine[10].Length == 1 ? "0" + htmlLine[10] : htmlLine[10],
            Admin2 = htmlLine[11],
            PlaceType = htmlLine[7],
            AlternativeNames = htmlLine[3].Split(',')
        };          

        return place;
    }
}

class App
{
    static void Main()
    {
        var placeFactory = new PlaceFactory(@"C:\Desenv\Texto.txt");

        Place place;

        place = placeFactory.ObterPlace(2);
        place = placeFactory.ObterPlace(0);
    }
}