Stream that reads byte per byte?

I created a simple method that reads byte-by-byte in a stream, but it is very slow and takes minutes to process a ten-megabyte file, I do not know why it is so slow. It must be byte-by-byte and not byte[] because it is an operation that I do in a single byte in a certain position and writes the same in the same position as the stream. The gambiarra code:

public void EncryptStream(Stream s) {

    // verificações
    if (s == null) throw new ArgumentNullException("s", "Stream cannot be nothing.");
    if (s.CanRead == false) throw new Exception("Stream is not readable.");
    if (s.CanWrite == false) throw new Exception("Stream is not writeable.");

    long len = s.Length;
    for(long i = 0; i <= len; i++) {
        s.Position = i;
        // "a" é o byte que está sendo operacionado no stream
        // "i" é a posição que o byte está no stream
        // o byte "a" será escrito novamente na mesma posição após a
        // operação abaixo.
        int a = s.ReadByte(); s.Position--;
        int pos = computePos(key, i);
        a += pos;
        s.WriteByte((byte)a);
    }
    byte checksun = performKeyHash(key);
    Console.WriteLine("CheckE = " + checksun.ToString());
    s.Position = len + 1;
    s.WriteByte(checksun);
    s.Close();
}

What's wrong to be so slow? Is there anything that can be done? Remembering that I can not cause Stream to read a byte buffer, it must be byte-by-byte .