Delete recyclerView item from Adapter

I'm trying to apply a feature by clicking an icon inside a list of a RecyclerView. By clicking on the icon, I register the acceptance of the condition, if not accepted, the "whole item" should be excluded from the list. I'm trying to do this from inside the Adapter, I believe it's in onBindViewHolder , because I want to click only on the item's icon, not the whole item, because I'll have other icons with other functions. I'm not able to delete the item from the list:

XML Adapter:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:gravity="fill_vertical"
    android:orientation="vertical"
    android:paddingLeft="@dimen/dim_mid"
    android:paddingRight="@dimen/dim_mid"
    android:paddingTop="@dimen/dim_normal">

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_marginBottom="@dimen/dim_normal"
        android:orientation="horizontal">

        <ImageView
            android:id="@+id/civUsuario"
            style="@style/ImgPerfil"
            android:layout_width="50dp"
            android:layout_height="50dp" />

        <LinearLayout
            android:layout_width="wrap_content"
            android:layout_height="match_parent"
            android:layout_weight="1"
            android:gravity="center_vertical"
            android:orientation="vertical"
            android:padding="5dp">

            <TextView
                android:id="@+id/textNome"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:text="Nome"
                android:textStyle="bold" />

            <TextView
                android:id="@+id/textStatus"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:text="Status" />
        </LinearLayout>

        <ImageView
            android:id="@+id/icone03"
            style="@style/IconesStatus" />

        <ImageView
            android:id="@+id/icone02"
            style="@style/IconesStatus" />

        <ImageView
            android:id="@+id/icone01"
            style="@style/IconesStatus" />

    </LinearLayout>

</LinearLayout>

Java Adapter:

List<Usuario> usuarios;

@Override
public void onBindViewHolder(@NonNull final MyViewHolder holder, final int position) {
    holder.icone01.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {

            // Executo a verificação
            if (false) {
                // Condição aceita
            } else {
                usuarios.remove(position);
                notifyItemRemoved(position);
            }

        }
    });
}

This code does not work ... the item is not deleted ... Can anyone tell me the correct way to do this? I need to run from within the Adapter to assign the onClick just to the specific icon, or access the icon, assigning an onClick to it from outside the Adapter, by recyclerView maybe ... I do not know how to do it.