You should use $(window).resize() to jQuery detect that the screen has been resized

Example:

$(function() {
    $(window).resize(function(){
      if ( $(this).width() < 760) {     
        $('#menu').css({'position' : 'relative'});
      }
    }
});

Update

After your upgrade, I imagine you're wanting something like this:

$(function() {
    $(window).scroll(function()
    {
        var topo = $('#topo').height(); // altura do topo
        var rodape = $('#rodape').height(); // altura do rodape
        var scrollTop = $(window).scrollTop(); // qto foi rolado a barra
        var tamPagina = $(document).height(); // altura da p?gina

        if($(window).width() > 760 && scrollTop > topo){
          $('#menu').css({'position' : 'fixed', 'top' : '0'});
        } else{
          $('#menu').css({'position' : 'relative', 'margin-top' : 0});
        }
    });
});

Because it must meet two conditions: The screen must be greater than 760px and must meet the desired condition when $(window).scrollTop() is greater than the top size

It is better to manipulate classes instead of elements directly.
You can start this menu with a class responsible for keeping it fixed at the top and then use Javascript / sup> only to check whether or not this class should be removed depending on the width of $(window) .

Already to handle resizing, use resize . Just check the width of the window and if the element has the class that holds the menu fixed. Example:

// js

$(function(){   
    if($(window).width() < 768){
        $('.menu').removeClass('is-fixed');
    }
});

$(window).resize(function(){
   var menu = $('.menu');

   /*
    * Se for menor que a largura que você definiu e...
    * Se o elemento possuir a classe 'is-fixed', então...
    * ela é removida.
    * 
    * Do contrário a classe é inserida.
    */
   if($(window).width() < 768 && menu.hasClass('is-fixed')){
       menu.removeClass('is-fixed');
   } else {
       menu.addClass('is-fixed');
   }
});

DEMO

I needed to put in JSFiddle because StackOverflow does not work with resize .

Try this, because if you do not want it to work if it is less than 760, use > in condition.

 $(window).scroll(function() {
     if ($(window).width() > 760)
         {
            $('#menu').css({'position' : 'absolute'});
         }
     });