ListView or Child Elements lose focus

I have a very simple ListView. Each row is tapable and loads a little custom PopupWindow when clicked. That works just fine.. most of the time.

The problem is, after quickly scrolling the ListView up and down a few times, the ListView seems to lose focus and no longer lets you tap on the rows. Leaving the activity and coming back resets the ListView and things work again... until you scroll quickly a few times.

The ListView is pretty simple:

<ListView android:id="@+id/favorites_list"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:listSelector="@drawable/list_selector_background"
/>

I've also tried setting android:focusableInTouchMode="true" and android:focusable="true" on the ListView - but that doesn't change anything. If I call list.requestFocusFromTouch(); after my list gets its data things work - but the ListView starts with the first row selected and it generally messes up the UI by selecting things randomly.

I'm listening for item clicks by implementing public void onItemClick(AdapterView<?> parent, View view, int position, long id) and setting the subclass of OnItemClickListener in the ListView via setOnItemClickListener(). Inside onItemClick I pop up a little PopupWindow - but that doesn't seem to be the problem here - the problem comes when I scroll the overall ListView up and down a few times.

How can I ensure that the elements in the list always are able to respond to touches?

Here is the adapter that gets used:

private class FavoritesAdapter extends BaseAdapter {

    private LayoutInflater mLayoutInflater;

    public FavoritesAdapter(){
        mLayoutInflater = (LayoutInflater) FavoritesActivity.this.getSystemService(LAYOUT_INFLATER_SERVICE);
    }

    @Override
    public int getCount() {
        return manager.getItemCount();
    }

    @Override
    public Object getItem(int arg0) {
        return null;
    }

    @Override
    public long getItemId(int arg0) {
        return 0;
    }

    @Override
    public View getView(int position, View convertView, ViewGroup viewGroup) {

        if(position == getCount() - 1)
            manager.retrieveNextPageOfFavorites();

        ViewHolder holder;

        if(convertView == null){
            convertView = mLayoutInflater.inflate(R.layout.favorites_list_item, null);

            holder = new ViewHolder();
            holder.brandName    = (TextView)convertView.findViewById(R.id.favorites_list_item_brand);
            holder.productName  = (TextView)convertView.findViewById(R.id.favorites_list_item_product);
            holder.size         = (TextView)convertView.findViewById(R.id.favorites_list_item_size);
            holder.width        = (TextView)convertView.findViewById(R.id.favorites_list_item_width);
            holder.image        = (ImageView)convertView.findViewById(R.id.favorites_list_item_image);


            ((TextView)convertView.findViewById(R.id.favorites_list_item_size_tag)).setTypeface(...);
            ((TextView)convertView.findViewById(R.id.favorites_list_item_width_tag)).setTypeface(...);

            convertView.setTag(holder);
        } else {
            holder = (ViewHolder)convertView.getTag();
        }

        FavoritesItemInfo fav = manager.getFavoritesItem(position);
        holder.brandName.setText(fav.brandName);
        holder.productName.setText(fav.productName);
        holder.size.setText(fav.size);
        holder.width.setText(fav.width);

        BitmapDrawable drawable = manager.getProductImage(position);
        holder.image.setImageDrawable(drawable);

        return convertView;
    }

    private class ViewHolder {
        public 开发者_如何学CTextView brandName;
        public TextView productName;
        public TextView size;
        public TextView width;
        public ImageView image;
    }
}