Link click function is not called

I already had a problem similar to this, and it would be because the function was built before the button was created. Now I have done a little bit after I give an append in my page I call a function on click.

This is my code on codepen: link

For you to understand the idea is that the person click the black buttons to add fields of the desired type, they set the name of the field and label in the middle div, and the third div is shown as this form.

What happens is that I want to be able to remove a field by clicking on the blue button, the problem is that this section is not called:

$('#remove_input').on("click", function() {
  var count_field = $('#count_field').val();
  var parent_class = $(this).parents('tr');
  if (parent_class.hasClass('pending')) {
    parent_class.remove();
    count_field--;
  } else if(parent_class.hasClass('saved')) {
    parent_class.removeClass('saved').addClass('remove');
  }
  $('#count_field').val(count_field);
});

This code is inside the function of clicking the add button after I make an append of the new field.

Can anyone help me?

Just to make clear the address field that already comes in html is with the correct operation that is just leave the button red. The problem is in the field added later by the add button.