Log System losing data, how to use Threads?

At times I needed a solution to something similar to your problem, not as big as but good-looking! The solution however was to use several threads to solve, as I am not a Delphi developer I had to buy a system and pay for its maintenance!

I've been posting the answer since I edited this question because I was looking for an old friend who solved the problem!

Follow the implementations he did for me!

Unit unt_funcaoLog

type
  {Classe responsável pela geração de LOG!}
  TGeraLog = Class
    private
      FCritical : TCriticalSection;

      Const C_ARQUIVO = '.\arquivo_log.txt';
      {Indica o arquivo em que o LOG será gerado, no caso, no mesmo diretório do executável!}
      Class var FInstance : TGeraLog;
      {Instância única do objeto!}
      Class function GetInstancia: TGeraLog; static;
      {Método responsável por instanciar o objeto singleton!}
    public
      procedure AfterConstruction; override;
      procedure BeforeDestruction; override;
      procedure GeraLog(Const AReferencia: String; Const AData: TDateTime; Const ATextoLog: String);
      {AReferencia..: Referência à Thread chamadora.}
      {AData........: Data e Hora da geração do LOG.}
      {ATextoLog....: Texto a ser escrito no arquivo de LOG.}
      function TryGeraLog(Const AReferencia: String; Const AData: TDateTime; Const ATextoLog: String): Boolean;
      {Método que TENTA gerar uma nova linha de LOG!}
      {AReferencia..: Referência à Thread chamadora.}
      {AData........: Data e Hora da geração do LOG.}
      {ATextoLog....: Texto a ser escrito no arquivo de LOG.}
      Class property Instancia: TGeraLog read GetInstancia;
      {Referência à instância singleton!}
  end;

implementation

{ TGeraLog }

procedure TGeraLog.AfterConstruction;
begin
  inherited;
  DeleteFile(Self.C_ARQUIVO);
  Self.FCritical := TCriticalSection.Create;
end;

procedure TGeraLog.BeforeDestruction;
begin
  inherited;
  Self.FCritical.Free;
end;

procedure TGeraLog.GeraLog(const AReferencia: string; const AData: TDateTime; const ATextoLog: string);
var
  _arquivo   : TextFile;
  sNovaLinha : String;
begin
  sNovaLinha := Format('%s|%s|%s', [AReferencia, DateTimeToStr(AData), ATextoLog]);

  {Entra na seção crítica!}
  Self.FCritical.Enter;
  try
    AssignFile(_arquivo, Self.C_ARQUIVO);
    if (FileExists(Self.C_ARQUIVO)) then
    begin
      Append(_arquivo);
    end
    else
    begin
      Rewrite(_arquivo);
    end;

    Writeln(_arquivo, sNovaLinha);

    CloseFile(_arquivo);
  finally
    {Sai da seção crítica}
    Self.FCritical.Release;
  end;
end;

Class function TGeraLog.GetInstancia: TGeraLog;
begin
  if not(Assigned(FInstance)) then
  begin
    FInstance := TGeraLog.Create;
  end;
  Result := FInstance;
end;

function TGeraLog.TryGeraLog(const AReferencia: String; const AData: TDateTime;
  const ATextoLog: String): Boolean;
var
  _arquivo   : TextFile;
  sNovaLinha : String;
begin
  sNovaLinha := Format('%s|%s|%s', [AReferencia, DateTimeToStr(AData), ATextoLog]);

  {Tenta entrar na seção crítica!}
  Result := Self.FCritical.TryEnter;
  if (Result = True) then
  begin
    try
      AssignFile(_arquivo, Self.C_ARQUIVO);
      if (FileExists(Self.C_ARQUIVO)) then
      begin
        Append(_arquivo);
      end
      else
      begin
        Rewrite(_arquivo);
      end;

      Writeln(_arquivo, sNovaLinha);

      CloseFile(_arquivo);
    finally
      {Sai da seção crítica}
      Self.FCritical.Release;
    end;
  end;
end;

initialization

finalization

  TGeraLog.Instancia.Free;

end.

Unit unt_diversasThreads

type
  TDiversaThread = Class(TThread)
    private
      FReferencia    : String;
      FDescricaoErro : String;

    public
      procedure Execute; override;
      {Rotina a ser executada pelo Thread que eventualmente gerará uma linha no arquivo de LOG!}

      property Referencia: String read FReferencia write FReferencia;
      {Referência que será escrito no arquivo de LOG para sabermos de onde veio a linha!}

      property DescricaoErro: String read FDescricaoErro;
  end;

implementation

{ TDiversasThread }

procedure TDiversaThread.Execute;
var
  bGeraLog : Boolean;
begin
  inherited;
  try
    {Loop enquanto o Thread não for finalizado!}
    while not (Self.Terminated) do
    begin
      {Aqui definimos um time para diminuir o consumo de CPU}
      Sleep(10);

      {Sorteia um número e verifica se o resto da divisão por dois é zero!}
      bGeraLog := (Random(1000000) mod 2) = 0;

      if (bGeraLog = True) then
      begin
        {Chama o método de geração de LOG!}
        TGeraLog.Instancia.GeraLog(Self.FReferencia, Now, 'O rato roeu a roupa do Rei de Roma');
      end;
    end;
  except
    on E: EInOutError do
    begin
      Self.FDescricaoErro := Format('Erro de I/O #%d - %s', [E.ErrorCode, SysErrorMessage(E.ErrorCode)]);
    end;
    on E: Exception do
    begin
      Self.FDescricaoErro := Format('(%s) - %s', [E.ClassName, E.Message]);
    end;
  end;
end;

end.

I'm writing logs to this call:

TGeraLog.Instancia.GeraLog('QUEM_ENVIOU_LOG', Now, 'TEXTO_QUE_DESEJA');

According to my friend the TryGeraLog function is more efficient when there is multiple access to the same file, but since it is not my case I only use GeraLog.

As far as I can tell, your system must be running on more than one computer simultaneously. In these cases, attempting to write to file by more than one process results in loss of data due to file locking. Another worsening is that when a file is opened by a share it takes longer to be freed up for writing to another computer on the network. Even worse if it's samba ...

So you should always use a more efficient mechanism for writing log. I would use a MESSAGE QUEUE style service to write the same log from different sources. A table in the Database would solve this and make it much easier to consult the LOG occurrences in the future.

Another way to write a single log from different sources is to use the Windows logging engine that is able to write to the event log also making the search easier.

This article is very nice to see how to make a write mechanism write to a remote computer (server)

To write to the local system logo, you can use the code below

uses SvcMgr; // Adicione ao seu Uses 

procedure frmFuncoes.GravaLogDetalhado(Texto: String, EventType: DWord = 1);
begin
  with TEventLogger.Create('NOME_APLICACAO') do
  begin
    try
      LogMessage(Texto, EventType);
    finally
      Free;
    end;
  end;
end;

and your code would use

procedure frmDetCliente.FormCreate(Sender: TObject);
begin
  GravaLogDetalhado('Abriu Detalhes do Clientes', EVENTLOG_INFORMATION_TYPE);
  ...
  ...//Blocos e blocos de códigos
  ... 
  GravaLogDetalhado('Clicou em Editar Cliente', EVENTLOG_INFORMATION_TYPE);
  ...//Blocos e blocos de códigos
  ...
  GravaLogDetalhado('Fechou Detalhes do Clientes', EVENTLOG_INFORMATION_TYPE);
end;

This way you pass the write control to the System.

I do not believe that Threads for each event would be a solution, since several departments will be able to access the same log file at the same time. I think there were other problems.

The best output in my opinion would be for each department to create its own log during a process and at the end of this process a thread is created to mount a general log using the data from this local log.

It would look more or less in that order:

1 - Process initiated by the user and all actions will be saved in a local file.

2 - At the end of the process create a thread for exclusive access to the "global" log file.

An example would be

begin
  if FileExists(Filename) then
    fstm := tFileStream.Create(Filename, fmOpenReadWrite or fmShareExclusive)
  else
    fstm := tFileStream.Create(Filename, fmOpenReadWrite or fmShareExclusive or fmCreate) end;

This would ensure that only one thread will access the file.

You can control the other threads to wait for access to the file.

The best solution in this case is to develop a separate system that listens to a port via UDP on the computer where the log file is. When it receives a log registration request, this application places this request in a waiting queue, whereas a TTimer checks the list from time to time and writes the requisitions in the correct order.

In your main application you put a procedure that makes this call to the UDP logging application through the server IP. It's a simple alternative and will do away with all file locking and data loss issues.

I think perhaps a simpler solution is to write this log to a table in the main database, or even to a separate database, depending on the volume of data that will be generated by these logs. From what I understood there would be several users running their executable on different computers, right? In that case, using threads to protect access to the file would not help much, no.

I do not advise you to use Threads (my opinion), after using several I come to the conclusion that it is not worth it because you can not debug it is when sometimes they stop working without a logical explanation, the best thing is you have a unit of functions where processes are done without harming open screen manipulation. I'm posting an example of how you could solve your problem.

procedure TForm1.Button2Click(Sender: TObject);
begin
 GravaLogDetalhado('Almoxarifado','Abriu Detalhes do Clientes');
end;

procedure TForm1.GravaLogDetalhado(Departamento,Texto: String);
 var
  Arquivo  : String;
  BkpLog   : TextFile;
begin
  Arquivo  := 'C:\Test.txt';

 if not(FileExists(Arquivo)) then
    begin
    AssignFile(BkpLog,Arquivo );
    //Um novo arquivo em disco com o nome atribuído à variável "Arquivo" é
    //criado e preparado para processamento, um ponteiro de arquivo é colocado no início do arquivo criado.
    //Caso o arquivo a ser criado já exista, esta instrução apaga o arquivo para criá-lo novamente.
    Rewrite(BkpLog);
    CloseFile(BkpLog);
     end
  else
    begin
        AssignFile(BkpLog, Arquivo);
      //Abre um arquivo texto existente para estendê-lo (saídas adicionais). Caso o arquivo não exista,
      //esta instrução provocará um erro de Entrada/Saída (Input/Output).
        Append(BkpLog);
        Writeln(BkpLog, Departamento + ' - ' + DateTimeToStr(now) + ' : ' + Texto );
        CloseFile(BkpLog);
  end;
end;