Questions about model improvement and misidentification

[pk429]

Thank you very much for your efforts. The PET model performed very well in our experiments, but we found that the model would misrecognize the back seats of electric vehicles, stone piers, and objects similar to heads. We tried to add similar negative samples to the data set for reinforcement, but with little effect. Is there a better improvement method or data set addition strategy?

[cxliu0]

Training with negative samples (i.e., images only contain similar objects) is a possible way to alleviate false positives. There are two ways you may try:

• Increase the number of negative samples;
• Perform compulsory sampling during training, ensuring that each batch contains at least one negative sample.

In practice, the second way may be more effective for reducing false positives.

[pk429]

Thank you very much for your reply. If my batchsize is set to 8, what is the appropriate ratio of negative samples to positive samples? Because I saw that random cropping of patches and random flipping are performed during data preprocessing.
If there is a background in my positive sample, there is a probability that it will be cropped into my headless background. On the contrary, the head image will not be collected in the negative sample no matter how it is cropped.
Therefore, how to balance this ratio, because in addition to the official data set, my data set will also have self-labeled data samples to improve generalization.

[cxliu0]

You may first try to sample one negative sample per batch, i.e., 7 positive samples and 1 negative sample in a batch of 8. In addition, to increase the probability that the patches cropped from positive samples contain objects, performing random cropping multiple times could be an alternative.

[pk429]

You may first try to sample one negative sample per batch, i.e., 7 positive samples and 1 negative sample in a batch of 8. In addition, to increase the probability that the patches cropped from positive samples contain objects, performing random cropping multiple times could be an alternative.

Thank you very much for your answer. I used the method you mentioned to ensure that there are negative samples participating in each batch. However, when I added negative samples, the model's recognition effect on blurred and crowded crowds dropped sharply, and it no longer actively splits points, which is similar to killing the query splitting ability.In addition, I added a background penalty mechanism to def loss_labels in the pet.py file, but it has not effectively improved the problem of blurring crowded crowds without splitting after adding negative samples.
########################################################################################
with torch.no_grad():

            pred_prob = torch.softmax(src_logits, dim=-1)[..., 1]  # [B, N] The probability of predicting "head"

        
        unmatched = (target_classes == 0) # Unmatched query background

       
        hard_neg_mask = (pred_prob > 0.5) & unmatched  

        
        hard_neg_weight = 3.0 # Penalty coefficient penalty background
       
        
        raw_ce_loss = raw_ce_loss * (1.0 + hard_neg_mask.float() * hard_neg_weight) # Perform weighting processing on raw_ce_loss

[cxliu0]

Based on your description, the performance drop could be attributed to the change of training patches. If the patch cropped from positive samples only contains a few people, then there is no need to split the queries. To alleviate this issue, you may try to crop each positive sample multiple times, ensuring that the cropped patch contains a certain number of people. The balance between positive samples and negative samples is worth noticing.

Regarding background penalty, given that the training patches do not contain a sufficient number of people, simply changing the loss function may not bring meaningful performance gain.