how to do consultation by Month?

  

(sic) I'm trying to make a query that returns the current month to me

Solution (assuming your db is MySQL or MariaDB ):

SELECT 'dia' FROM 'caixa' WHERE 'data' >= '2018-12-01' AND 'data' <= '2018-12-31';

In PHP you will only need to replace the month in the query , thus (following your example):

<?php session_start();
    date_default_timezone_set('America/Sao_Paulo');
    //$d = date('d');
    //$m = date('m');
    //$y = date('Y');
    //$query_cx = "SELECT dia FROM caixa WHERE MONTH(data)='$m' ORDER BY id DESC";
    $query_cx = "SELECT 'dia' FROM 'caixa' WHERE 'data' >= '".date('Y-m')."-01' AND 'data' <= '".date('Y-m')."-31';";
    $result_cx = mysqli_query($conectar, $query_cx);
    while ($linhas_cx = mysqli_fetch_assoc($result_cx)){
       // echo "$linhas_cx['valor']; // PQ USAR ASPAS AQUI (e ainda falta fechar)?
       echo $linhas_cx['valor'];
    }

  

1- Will this query end up returning me the current Month and the same month of the previous year?

No, because we declare the year in the query.

  

2- How would I search for some previous Month, if the year will be different?

Just change the query :

$query_cx = "SELECT 'dia' FROM 'caixa' WHERE 'data' >= '".date('Y-m', strtotime('-1 month'))."-01' AND 'data' <= '".date('Y-m', strtotime('-1 month'))."-31';";

Recommended reading :

  

Manipulating dates with PHP

A simple way to do this is to use only the sql itself

SELECT dia FROM caixa WHERE MONTH(data)= MONTH(NOW()) ORDER BY id DESC