Regarding Feature Maps

[vipingautam1906]

Hello Vladimir, Thank you very much for making your work publicly available .
I’m currently working in the domain of adversarial attacks and am very interested in experimenting with BPreID. I wanted to ask whether it's possible to export the full model or, alternatively, extract final layer feature maps during inference.
We plan to cite your work in our research, but at the moment, we’re unsure which part of the code provides this functionality. Any guidance or pointers would be extremely helpful.
Thank You

[VlSomers]

Hi @vipingautam1906. The final features maps (i.e. the output HxWxD feature map from the backbone), is located here. Do you want to extract this feature map on your custom images, or on images from popular reid datasets? You could also perform you experiments on KPR, where there is an easy to use demo to perform inference on any image. Let me know if I can do anything to help.

[MattLiutt]

Hi @VlSomers, I've checked both KPR and bpbreid and the demo script. If let say I wanna have feature extraction, do I simply put random image for inference with bpbreid? or I need to input keypoints as prompt like KPR? Thanks a lot!

[VlSomers]

Hi @MattLiutt , for BPBreID you can simply pass image, this model does not input ketpoints :-)

[MattLiutt]

@VlSomers Thanks for the reply! Does it mean that I only have to call the bpbreid model and then use feature extraction function for getting the embedding?
I simply refactor a bit based on demo.py and run like below but not sure if it's correct.

bpb_cfg = build_config(config_file="configs/bpbreid/bpbreid_market1501_test.yaml")
bpb_cfg.use_gpu = torch.cuda.is_available(

extractor = FeatureExtractor(bpb_cfg)

embedding = extractor('image.png')

However, it always gives error below:

Traceback (most recent call last):
  File "/home/bpbreid/./demo.py", line 50, in <module>
    embedding = extractor('/home/bpbreid/dataset/Market-1501-v15.09.15/bounding_box_test/-1_c1s1_021451_03.jpg')
  File "/home/bpbreid/torchreid/tools/feature_extractor.py", line 167, in __call__
    masks = result['mask'].unsqueeze(0).to(self.device)
KeyError: 'mask'

[VlSomers]

@MattLiutt Can you tell me a little bit more about what you are trying to achieve? Are you working with @vipingautam1906 ? You say you want to get the "embeddings", but then pointed out to the feature extraction function that return a feature map. Are you interested in the final output embeddings (i.e. the one dimensional part-based embedding, on per per body part), or are you interested in the HxWxD feature map output by the backbone? Also, I'm not sure demo.py from KPR would be compatible with the BPBreID codebase from this repository. Copy pasting your entire logs would also be helpful. Finally, I'm not sure the feature_extractor.py of bpbreid is working (as compared to the one in KPR) -> can you maybe try to commend out the line 167 that and simply set masks to None?

[MattLiutt]

Thanks! I'm not working with @vipingautam1906, I just search related topics in the issue and trying to see if similar issues are solved. I'm trying to see how good the model is doing reid. I'm trying to get the feature embedding and then compare their distance with different pairs of images (assuming they're same persons or objects). So I was trying to make use of feature_extractor.py to see if they're possible to extract feature maps. Or from your point of view of this task, what approaches would you suggest?

import json
import os
import cv2
import numpy as np
import torch

from pathlib import Path
# from torchreid.metrics.distance import compute_distance_matrix_using_bp_features
from torchreid.scripts.main import build_config, build_torchreid_model_engine
from torchreid.tools.feature_extractor import FeatureExtractor
# from torchreid.utils.visualization.display_kpr_samples import display_kpr_reid_samples_grid, display_distance_matrix


bpb_cfg = build_config(config_file="configs/bpbreid/bpbreid_market1501_test.yaml")
bpb_cfg.use_gpu = torch.cuda.is_available() # already done in build_config(...), but can be overwritten here
# bpb_cfg.model.promptable_trans.disable_inference_prompting = True  # Disable prompting during inference (keypoints prompts are ignored by KPR)

extractor = FeatureExtractor(bpb_cfg)
print("BPB Feature Extractor successfully initialized.")

embedding = extractor('/home/bpbreid/dataset/Market-1501-v15.09.15/bounding_box_test/-1_c1s1_021451_03.jpg')
print("Extracted embedding shape:", embedding.shape)

/home/bpbreid_env/lib/python3.10/site-packages/albumentations/__init__.py:24: UserWarning: A new version of Albumentations is available: 2.0.8 (you have 1.4.24). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
/home/bpbreid_env/lib/python3.10/site-packages/torchmetrics/utilities/imports.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import DistributionNotFound, get_distribution
Overwriting current config with config loaded from pretrained_models/bpbreid_market1501_hrnet32_10642.pth
Diff from default config :
{'height': 384, 'load_config': True, 'preprocess': 'five_v'}
building model on device cuda
=> init weights from normal distribution
Loading pretrained ImageNet HRNet32 model at pretrained_models/hrnetv2_w32_imagenet_pretrained.pth
=> loading pretrained model pretrained_models/hrnetv2_w32_imagenet_pretrained.pth
Model: bpbreid
- params: 34,855,494
- flops: 8,000,205,312
Building train transforms ...
+ resize to 256x128
+ normalization (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+ to torch tensor of range [0, 1]
Building test transforms ...
+ resize to 256x128
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+ masks preprocess = five_v
+ use add background mask
BPB Feature Extractor successfully initialized.
Preprocess result keys: None
Traceback (most recent call last):
  File "/home/bpbreid/./demo.py", line 21, in <module>
    embedding = extractor('/home/bpbreid/dataset/Market-1501-v15.09.15/bounding_box_test/-1_c1s1_021451_03.jpg')
  File "/home/bpbreid/torchreid/tools/feature_extractor.py", line 166, in __call__
    masks = result['mask'].unsqueeze(0).to(self.device)
KeyError: 'mask'

If I commented out the line, so I think we cannot comment it out.

/home/bpbreid_env/lib/python3.10/site-packages/albumentations/__init__.py:24: UserWarning: A new version of Albumentations is available: 2.0.8 (you have 1.4.24). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
/home/bpbreid_env/lib/python3.10/site-packages/torchmetrics/utilities/imports.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import DistributionNotFound, get_distribution
Overwriting current config with config loaded from pretrained_models/bpbreid_market1501_hrnet32_10642.pth
Diff from default config :
{'height': 384, 'load_config': True, 'preprocess': 'five_v'}
building model on device cuda
=> init weights from normal distribution
Loading pretrained ImageNet HRNet32 model at pretrained_models/hrnetv2_w32_imagenet_pretrained.pth
=> loading pretrained model pretrained_models/hrnetv2_w32_imagenet_pretrained.pth
Model: bpbreid
- params: 34,855,494
- flops: 8,000,205,312
Building train transforms ...
+ resize to 256x128
+ normalization (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+ to torch tensor of range [0, 1]
Building test transforms ...
+ resize to 256x128
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+ masks preprocess = five_v
+ use add background mask
BPB Feature Extractor successfully initialized.
Traceback (most recent call last):
  File "/home/bpbreid/./demo.py", line 21, in <module>
    embedding = extractor('/home/bpbreid/dataset/Market-1501-v15.09.15/bounding_box_test/-1_c1s1_021451_03.jpg')
  File "/home/bpbreid/torchreid/tools/feature_extractor.py", line 190, in __call__
    features = self.model(images, external_parts_masks=masks)
UnboundLocalError: local variable 'masks' referenced before assignment

[VlSomers]

That's because you need to set masks to None, please ask your favorite LLM for help on this 😉

[MattLiutt]

OK, so as long as the external_parts_masks=None, it should work right?

[VlSomers]

Exactly, this one is optional

[vipingautam1906]

Hi

Hi @vipingautam1906. The final features maps (i.e. the output HxWxD feature map from the backbone), is located here. Do you want to extract this feature map on your custom images, or on images from popular reid datasets? You could also perform you experiments on KPR, where there is an easy to use demo to perform inference on any image. Let me know if I can do anything to help.

The issue is resolved. We've managed to extract the features. Once again thank you so much for your response.

[MattLiutt]

@VlSomers sorry to bother you again, as I've tried, is it always return True for visibility score if external_parts_masks=None? Like what you mentioned here. I run the script to get bn_foreg and compare. However, it seems like the feature extracted from same image is always different. Do you encounter this situation before?

[VlSomers]

@MattLiutt it really depends on your image, if you have some occluded/invisible parts, it should not always return true for the visibility score. Moreover, the foreground visibility score will always be true (only the body parts embeddings might have False visibility scores). 'It seems like the feature extracted from same image is always different', this should not happen at all, maybe the weights did not load properly and therefore the model is randomly initialized, I would need to see your logs for that

[MattLiutt]

@MattLiutt it really depends on your image, if you have some occluded/invisible parts, it should not always return true for the visibility score. Moreover, the foreground visibility score will always be true (only the body parts embeddings might have False visibility scores). 'It seems like the feature extracted from same image is always different', this should not happen at all, maybe the weights did not load properly and therefore the model is randomly initialized, I would need to see your logs for that

I thought the reason is that I turn weights_only=False, but if I remain unchanged, there will be following log:

/home/bpbreid_env/lib/python3.10/site-packages/albumentations/__init__.py:24: UserWarning: A new version of Albumentations is available: 2.0.8 (you have 1.4.24). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
/home/bpbreid_env/lib/python3.10/site-packages/torchmetrics/utilities/imports.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import DistributionNotFound, get_distribution
Unable to load checkpoint from "pretrained_models/bpbreid_market1501_hrnet32_10642.pth"
Traceback (most recent call last):
  File "/home/bpbreid/./demo.py", line 13, in <module>
    bpb_cfg = build_config(config_file="configs/bpbreid/bpbreid_market1501_test.yaml")
  File "/home/bpbreid/torchreid/scripts/main.py", line 208, in build_config
    checkpoint = load_checkpoint(cfg.model.load_weights)
  File "/home/bpbreid/torchreid/utils/torchtools.py", line 88, in load_checkpoint
    checkpoint = torch.load(fpath, map_location=map_location)
  File "/home/bpbreid_env/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load
    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. 
        (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
        (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
        WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function.

Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.