I think you can change your logic a bit by applying the hint that @jean said in the comments, also changing the way you validate operations. I gave an example to explain better in the answer.

Screen construction

The print below contains a screen with two JTextField : one called fieldValorAnterior , which is smaller and is more at the top to display the partial account and another called fieldValorAtual bellow, pasted in fieldValorAnterior that will display the numbers as they are clicked.

Also contains numeric buttons, floating point, and basic operations.

Creatinglisteners

Assumingthatyournumericbuttonshaveaunitvalueof0to9,youcanmakeasinglelistenerforthem,whoseonlyfunctionistoconcatenatetheexistingvaluewiththenewclickedvalue:

classAcaoBotaoNumericoListenerimplementsActionListener{@OverridepublicvoidactionPerformed(ActionEvente){JButtonbtn=(JButton)e.getSource();StringvalorAtual=fieldValorAtual.getText();fieldValorAtual.setText(valorAtual+btn.getText());}}

JustapplythisActionListenertothebuttons:

this.btn0.addActionListener(newAcaoBotaoNumericoListener());//...this.btn9.addActionListener(newAcaoBotaoNumericoListener());

Alsoassumingyouhaveoperatorbuttonswhosetextisthecorrespondingoperator,justcreateaactionListenerthatwillhandlethecontentwhenanoperationbuttonisclicked:

classAcaoBotaoOperacaoListenerimplementsActionListener{@OverridepublicvoidactionPerformed(ActionEvente){JButtonbtn=(JButton)e.getSource();StringvalorAtual=fieldValorAtual.getText();StringvalorAnterior=fieldValorAnterior.getText();if(!valorAtual.isEmpty()&&!valorAnterior.isEmpty()){fieldValorAtual.setText("");
            fieldValorAnterior.setText(efetuarCalculo(valorAnterior, valorAtual) + " " + btn.getText());
        } else if (!valorAtual.isEmpty() && valorAnterior.isEmpty()) {
            fieldValorAtual.setText("");
            fieldValorAnterior.setText(valorAtual + " " + btn.getText());
        } else if (valorAtual.isEmpty() && !valorAnterior.isEmpty()) {
            fieldValorAnterior.setText(valorAnterior.substring(0, valorAnterior.length() - 1) + btn.getText());
        }
    }
}

Notice that the last condition changes the signal of the current operation, similar to the Windows calculator. The first condition will calculate and concatenate with the new operator clicked, since both textfields have values, and fieldAnterior will always be either empty or contain a number and a sign separated by space (so I explain why use space). The second condition will only concatenate the value with the operator and display in fieldAnterior .

To apply the action to the buttons:

    this.btnAdicao.addActionListener(new AcaoBotaoOperacaoListener());
    this.btnSubtracao.addActionListener(new AcaoBotaoOperacaoListener());
    this.btnMultiplicacao.addActionListener(new AcaoBotaoOperacaoListener());
    this.btnDivisao.addActionListener(new AcaoBotaoOperacaoListener());

Now the = button needs a listener the part:

this.btnIgual.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                String valorAtual = fieldValorAtual.getText();
                String valorAnterior = fieldValorAnterior.getText();
                if (!valorAtual.isEmpty() && !valorAnterior.isEmpty()) {

                    fieldValorAtual.setText(efetuarCalculo(valorAnterior, valorAtual));
                }
            }
        });

Note that the account will only be processed if it has values in both fields, if this condition is not met, we will not have 2 values and a signal to pass to class OperacaoAritmetica .

The class OperationAritmetica

Finally, the method responsible for passing these values to our class OperacaoAritmetica :

private String efetuarCalculo(String valorAnterior, String valorAtual) {
    String valores[] = (valorAnterior + " " + valorAtual).split("\s");

    OperacaoAritmetica operacao = new OperacaoAritmetica(valores[1], Double.parseDouble(valores[0]), Double.parseDouble(valores[2]));
    fieldValorAnterior.setText("");
    String resultado;
    try {
        double res = operacao.calcular();
        resultado = res % 1 == 0 ? Math.round(res) + "" : res + "";
    } catch (ArithmeticException ex) {
        resultado = "!ERROR";
    }

    return resultado;
}

Condition res % 1 == 0 checks whether the returned double is an exact integer, and converts to int , just so that an unexpected zero does not appear as in 1 + 3 = 4.0. I used the split so that it was possible, in a simpler way, to separate the 3 values (value 1, value 2 and operator), so space was used in listener of the operation buttons.

To perform operations (and following the idea already passed in the comments), you can create a class that receives the operator's signal and the two values to be calculated, and in that class, implement all 4 operations (careful with division by zero!) that will be our class OperacaoAritmetica :

public class OperacaoAritmetica {

    private final String SOMA = "+";
    private final String SUBTRACAO = "-";
    private final String MULTIPLICACAO = "*";
    private final String DIVISAO = "/";

    private String operacao;
    private double valor1;
    private double valor2;

    public OperacaoAritmetica(String sinal, double valor1, double valor2) {

        this.valor1 = valor1;
        this.valor2 = valor2;
        this.operacao = sinal;
    }

    public double calcular() {

        switch (operacao) {
            case SOMA:
                return somar(valor1, valor2);
            case SUBTRACAO:
                return subtrair(valor1, valor2);
            case MULTIPLICACAO:
                return multiplicar(valor1, valor2);
            case DIVISAO:
                return dividir(valor1, valor2);
            default:
                throw new IllegalArgumentException("Operador não é válido");

        }
    }

    private double somar(double valor1, double valor2) {
        return valor1 + valor2;
    }

    private double subtrair(double valor1, double valor2) {
        return valor1 - valor2;
    }

    private double multiplicar(double valor1, double valor2) {
        return valor1 * valor2;
    }

    private double dividir(double valor1, double valor2) {
        if (valor2 == 0) {
            throw new ArithmeticException("Não é possivel dividir por zero.");
        }
        return valor1 / valor2;
    }
}

If you want to further refine that floating-point button, just add listener like this:

     this.btnPontoSeparador.addActionListener(new ActionListener() {
        @Override
        public void actionPerformed(ActionEvent e) {
            String strValorAtual = fieldValorAtual.getText();
            String strpontoFlutuante = ((JButton) e.getSource()).getText();
            if (!strValorAtual.contains(".") && strValorAtual.isEmpty()) {
                strValorAtual = "0" + strpontoFlutuante;
            } else if (!strValorAtual.contains(".") && !strValorAtual.isEmpty()) {
                strValorAtual += strpontoFlutuante;
            }
            fieldValorAtual.setText(strValorAtual);
        }
    });

If the value is blank, it will fill with 0. , if it is not, it will only concatenate the same point.

And also the Clear button:

    this.btnLimpar.addActionListener(new ActionListener() {
        @Override
        public void actionPerformed(ActionEvent e) {
            fieldValorAnterior.setText("");
            fieldValorAtual.setText("");
        }
    });
}

With this you can already have a sense of how to create a pretty basic calculator similar to windows. Of course this example did not address how to recover a negative value, among other things that could be done, but it was just an example, the rest you can break your head and try to implement.

This link is a complete and functional example that can be tested.