This is what session tracking is for. Turn it on. It should automatically lock the pcap filter tighter for you.