What is with in Python?

22

I'm giving a study on Python and I came across this with .

I know it from javascript, but I do not know how it works on Python

Most of the examples I've seen show it being used in reading files.

Example:

with open("my_file.txt") as file:
    data = file.read()
    #faça algo com "data"
    
asked by anonymous 03.02.2015 / 15:57

3 answers

29

It is used to guarantee finalization of purchased assets.

A file, for example, opens. Who guarantees that it will be closed? Even if you explicitly put it in the code that it should be closed, if an exception occurs, the code exits without executing the rest of the code that is scoped, it skips the closure.

To avoid this we use try finally . finally ensures completion. As the code gets a bit long and this case is quite frequent the language has provided a simplified form with with .

It can handle objects that contain methods __enter__() and __exit__() . They are called internally right at the beginning of the created block execution and within the% internal% created in the block.

In the example quoted there should look something like this internally:

try:
    __enter__()
    open("my_file.txt") as file:
        data = file.read()
        #faça algo com "data"
finally:
    __exit__()

Certainly within finally has __exit__() to close the file. About close() you already know how it works according to this question you made earlier. The operation in PHP and Python as in almost any language that has this feature is virtually identical.

To help what was said in the comments, it is analogous but not identical to the finally of C # and VB.Net (not to be confused with the directive) or using with resources of Java (not to be confused with try ()

Documentation .

    
03.02.2015 / 16:04
15

The with command is used to make it easier to write any block of code that involves features that need to be "finalized" (that is, restored, released, closed, etc.) after the block is closed - and it allows this to be done automatically, with the termination logic within the object used.

Then first thing: the with end code always runs - it does not matter if an error occurred within the with block or not.

But this could already be done with block finally and a try...finally clause - it does not matter the result of the block within try , finally always runs.

The great differential of with is that the programmer who is using objects that need finalization does not have to worry about calling these methods explicitly.

In the case of a file, people generally know that it has to be called close - but the most common, since close already is used is that nobody ends up programming a snippet that manipulates file with call to close , without using with remembering to put this within finally . That is, with encourages the code to be written in a way that behaves correctly:

wrong way:

f = open("data.txt", "w")
# codigo usando o arquivo f
f.close()

The right way, without the with:

try:
   f = open("data.txt", "w")
   # codigo usando o arquivo f
finally:
   f.close()

right way with (recommended):

with open("data.txt", "w") as f:
   # codigo usando o arquivo f

(the file is automatically closed at the end of the block).

Then note that the "right" code is still a shorter line than the "wrong" one. But - as I mentioned above, close of files is not the coolest thing of with - why everyone knows and remembers close (hopefully) - but is that for any object that needs a finalization - thread locks, database transactions, change the state of the terminal to read keys without requiring <enter> , monkey patch of functions and methods in tests, with executes completion transparently - without user needing know if you need to call the method close , release , undo of each object (in fact, it forces the programmers of the classes of the objects that work with to implement a common interface, with methods __enter__ and __exit__ .

Furthermore, when you are programming your own classes that can be used with with , you can treat within your function code __exit__ the most common or expected errors that could happen in the block: __exit__ receives information about the exception that occurred inside the block, and __exit__ can choose to handle the error and suppress the exception, or simply release the resource and pass the error forward: ie a lot of code that would always be the even if someone would have to write within the except all clause when using their object can stay within __exit__ and people use their object without worrying about the code that would go inside except .

Here is a simple example of a "bank account" that makes transactions with the balance but only allows transactions to take effect at the end of the with block if there is a positive balance in all accounts:

class ContaBancariaTransacional:
    def __init__(self, titular, saldo_inicial=0):
        self.titular = titular
        self.saldo = saldo_inicial

    def deposito(self, valor):
        self.temp += valor

    def retirada(self, valor):
        if self.saldo - self.temp - valor < 0:
            raise ValueError("Não há saldo suficiente")
        self.temp -= valor
        return valor

    def __enter__(self):
        self.temp = 0

    def __exit__(self, tipo_excecao, valor_excecao, traceback):
        if tipo_excecao is None:
            # Não ocorreu nenhum erro no bloco, e podemos
            # atualizar o saldo definitivo
            self.saldo += self.temp
        del self.temp

    def __repr__(self):
        return f"conta de {self.titular}. Saldo: {self.saldo:.02f}"

def transferir(conta1, conta2, valor):
    with conta1, conta2:

        conta2.deposito(conta1.retirada(valor))

And in an interactive session we can see:

In [96]: conta1 = ContaBancariaTransacional("João", 100)

In [97]: conta2 = ContaBancariaTransacional("José", 0)

In [98]: conta1, conta2
Out[98]: (conta de João. Saldo: 100.00, conta de José. Saldo: 0.00)

In [99]: transferir(conta1, conta2, 100)

In [100]: conta1, conta2
Out[100]: (conta de João. Saldo: 0.00, conta de José. Saldo: 100.00)

In [101]: transferir(conta1, conta2, 50)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
(...)

ValueError: Não há saldo suficiente

In [102]: conta1, conta2
Out[102]: (conta de João. Saldo: 0.00, conta de José. Saldo: 100.00)

Disclaimer : This code is not complete to be a "two phase transaction" - and in fact, it only works because the account where the balance comes out is before in the with command - when __exit__ of the first account is called, it raises the exception, which then prevents the balance increase in account2. Also, to put into production it would be nice to have some locks per thread in that class of Account as well, no matter how much toy it may be. But I think you can see how with can work with classes defined by the programmer himself.

Now, without being for transfer, imagine that this object "account" is used by a system that is inside a loot machine (in case access to the object account would be remote, with some remote method call protocol, of course - but assuming a Python library to operate the machine's hardware - the code for withdrawal could simply be:

def saque(atm, conta, valor):
    with conta:
        valor = conta.retirada(valor)
        atm.disponibilizar_notas(valor)
        atm.abrir_gaveta()
        atm.esperar(10)
        atm.fechar_gaveta()
        if not atm.gaveta_vazia():
            raise RuntimeError("Usuario nao retirou o dinheiro")

Ready - If any of the methods involving mechanical parts of the machine even make an error, the actual balance of the account is not debited. (Note that if there is no desired value in the account, the withdrawal method itself already aborts the transaction.)

    
28.04.2017 / 00:41
11

Syntax

The syntax of the with statement is:

with EXPR as VAR:
    BLOCK

with and as are reserved words of the language. EXPR is an arbitrary expression (but not a list of expressions) and VAR is a single target for assignment (an object, in a simplified way). VAR can not be a list of variables separated by commas, but can be a list of variables separated by commas in parentheses (will be explained later). The as VAR part is optional, so the syntax below is still valid:

with EXPR:
    BLOCK

From version 2.7 of the language you can use multiple context managers:

with EXPR1 as VAR1, EXPR2 as VAR2:
    BLOCK

Which is analogous to creating nested with statements:

with EXPR1 as VAR1:
    with EXPR2 as VAR2:
        BLOCK

Operation

The order of execution is:

  • The expression given by EXPR is evaluated in order to get the context manager;
  • The __exit__ method of the context manager is loaded for future use;
  • The __enter__ method of the context manager is called;
  • If VAR is set, VAR is returned by method __enter__ ;
  • The code block defined by BLOCK is executed;
  • The __exit__ method is called. If any exceptions were triggered by BLOCK , the type ( type ), value ( value ), and traceback information of the exception are passed as a parameter. Otherwise, if BLOCK is executed without generating errors, it is terminated with the statements break , continue or return , the __exit__ method will be executed with all three parameters equal to None . >

    The closest translation of the first statement about its operation is:

    # 1. A expressão é avaliada para obter o gerenciador de contexto:
    mgr = (EXPR)
    
    # 2. O método __exit__ é carregado
    exit = type(mgr).__exit__
    
    # 3. O método __enter__ é executado:
    value = type(mgr).__enter__(mgr)
    
    exc = True
    try:
        try:
            # 4. O retorno de __enter__ é atribuído a VAR:
            VAR = value
    
            # 5. O bloco de código é executado:
            BLOCK
        except:
            # Captura os erros ocorridos durante a execução de BLOCK:
            exc = False
    
            # 6. Executa o método __exit__ passando as informações de erro:
            if not exit(mgr, *sys.exc_info()):
                raise
    finally:
        # 6. Executa o método __exit__ passando None para os parâmetros:
        if exc:
            exit(mgr, None, None, None)
    

    The execution details may vary depending on the characteristics of the objects involved.

    If the expression given in EXPR does not have at least one of the __enter__ and __exit__ methods, a AttributeError exception will be thrown, first to __exit__ , second to __enter__ . p>

    Example

    Consider the example:

    class Foo (object):
    
        def __enter__ (self):
            print("Enter")
            return 1
    
        def __exit__ (self, exit, value, exc):
            print("Exit")
    
    with Foo() as foo:
        print(foo)
    

    The output of the code will be:

    Enter
    1
    Exit
    
      

    Code working on Ideone .

    The execution is the same as:

    import sys
    
    mgr = Foo()
    exit = Foo.__exit__
    value = Foo.__enter__(mgr)
    exc = True
    try:
        try:
            foo = value
            print(foo)
        except:
            exc = False
            if not exit(mgr, *sys.exc_info()):
                raise
    finally:
        if exc:
            exit(mgr, None, None, None)
    
      

    Code working on Ideone .

    If the __enter__ method returns a list of values:

    class Foo (object):
    
        def __enter__ (self):
            print("Enter")
            return 1, 2, 3
    
        def __exit__ (self, exit, value, exc):
            print("Exit")
    

    You can assign a comma-separated list of parentheses within the with declaration:

    with Foo() as (a, b, c):
        print(a, b, c)
    

    The output would be:

    Enter
    1 2 3
    Exit
    
      

    Code working on Ideone .

    Compatibility

    The declaration of with was added in version 2.5 of the language, however, it is necessary to enable it through the code:

    from __future__ import with_statement
    

    As of version 2.6, the declaration is already enabled by default. Versions prior to 2.5 are not supported by this statement.

    Context Managers

    You can practically define new context managers from the contextlib library, using the contextmanager decorator.

    from contextlib import contextmanager
    
    @contextmanager
    def tag(name):
        print("<%s>" % name)
        yield
        print("</%s>" % name)
    
    with tag("h1"):
        print("Stack Overflow")
    

    The output generated is:

    <h1>
    Stack Overflow
    </h1>
    

    The operation is analogous considering what is before yield as the __enter__ method and what is after the __exit__ method (only for result merit, but are NOT analogous).

    Examples

    The lower examples have been taken from PEP 343 itself, they are not entirely functional, but they express very well the use of, but not limited to, the statement in question.

  • When the resource used needs to be blocked at the beginning of the run and released at the end of the run:

    @contextmanager
    def locked(lock):
        lock.acquire()
        try:
            yield
        finally:
            lock.release()
    
    with locked(myLock):
        # Código executado com myLock bloqueado. O recurso será
        # liberado quando o bloco de código ser executado.
    
  • When it is necessary for a file opened by the program to be closed at the end of the program:

    @contextmanager
    def opened(filename, mode="r"):
        f = open(filename, mode)
        try:
            yield f
        finally:
            f.close()
    
    with opened("/etc/passwd") as f:
        for line in f:
            print line.rstrip()
    
  •   

    This behavior is already native to the open function of Python.

  • When working with transactions in the database:

    @contextmanager
    def transaction(db):
        db.begin()
        try:
            yield None
        except:
            db.rollback()
            raise
        else:
            db.commit()
    
  • When you want to temporarily redirect the output target:

    @contextmanager
    def stdout_redirected(new_stdout):
        save_stdout = sys.stdout
        sys.stdout = new_stdout
        try:
            yield None
        finally:
            sys.stdout = save_stdout
    
    with opened(filename, "w") as f:
        with stdout_redirected(f):
            print("Hello world") # Será escrito no arquivo filename
    
  • Other examples can be found in the other answers to this question, in PEP 343 or other references.

    References:

    PEP 343 - The "with" Statement

    Compound statements - The with statement

        
    28.04.2017 / 00:15