[Qwen3-VL] feature-extraction export fails: unordered_map::at crash due to torch.vmap in masking logic

[OntosAI]

I am attempting to export the Qwen3-VL-Embedding-2B model to OpenVINO. This blocks the deployment of the new multimodal embedding capabilities.
I have encountered two blockers:
Optimum-CLI limitation: feature-extraction task is not yet supported for qwen3_vl architecture.
OpenVINO conversion crash: Manual conversion using ov.convert_model fails with RuntimeError: unordered_map::at. The traceback indicates an issue tracing torch.vmap operations within transformers.masking_utils (specifically _vmap_for_bhqkv), even when attn_implementation="eager" is explicitly set.
To Reproduce

Environment:
openvino==2025.3
torch==2.8.0
transformers (Qwen3-VL branch/latest)
optimum-intel(git+https://github.com/openvino-dev-samples/optimum.git@qwen3vl)
Minimal Reproduction Script:
code
Python :
import torch
import openvino as ov
from transformers import AutoModel, AutoProcessor
from PIL import Image

model_id = "Qwen/Qwen3-VL-Embedding-2B"

Load with eager attention to attempt disabling FlashAttn/SDPA optimization

model = AutoModel.from_pretrained(
model_id,
trust_remote_code=True,
attn_implementation="eager",
device_map="cpu"
)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

Prepare multimodal dummy input

dummy_image = Image.new('RGB', (28, 28), color='black')
dummy_text = "<|image_pad|>Describe this image."
inputs = processor(text=[dummy_text], images=[dummy_image], return_tensors="pt")

Wrapper to align with OpenVINO input expectations

class Wrapper(torch.nn.Module):
def init(self, model):
super().init()
self.model = model
def forward(self, input_ids, attention_mask, pixel_values, image_grid_thw):
return self.model(
input_ids=input_ids, attention_mask=attention_mask,
pixel_values=pixel_values, image_grid_thw=image_grid_thw,
output_hidden_states=True
).last_hidden_state

Crash happens here

ov_model = ov.convert_model(
Wrapper(model),
example_input=(inputs.input_ids, inputs.attention_mask, inputs.pixel_values, inputs.image_grid_thw)
)
Relevant Traceback
The error occurs deep within the PyTorch frontend when handling the vectorized masking logic:
Traceback (most recent call last):
...
File ".../transformers/masking_utils.py", line 392, in sdpa_mask_recent_torch
causal_mask = _vmap_for_bhqkv(mask_function)(batch_arange, head_arange, cache_position, kv_arange)
...
File ".../torch/_functorch/vmap.py", line 484, in _flat_vmap
batched_outputs = func(*batched_inputs, **kwargs)
...
File ".../openvino/frontend/pytorch/ts_decoder.py", line 84, in init
raise RuntimeError(
RuntimeError: Couldn't get TorchScript module by tracing.
Exception:
unordered_map::at
Request
Please add support for task="feature-extraction" for qwen3_vl in Optimum Intel.
Fix the OpenVINO PyTorch frontend to handle (or correctly bypass) torch.vmap / functorch constructs used in the new Transformers masking implementation, or provide a workaround to strictly disable these paths during tracing.

[rutiktambe]

This looks like two separate issues at different layers as follows :

1. Optimum Intel: qwen3_vl does not currently support task="feature-extraction", which blocks exporting multimodal embedding models via optimum-cli. This should be fixable via task registration/mapping. This can be fixed by registering the architecture for feature extraction :

# Optimum Intel – task registration
FEATURE_EXTRACTION_SUPPORTED_ARCHS.add("qwen3_vl")

2. OpenVINO PyTorch frontend: Conversion fails due to torch.vmap / functorch usage in the new Transformers masking path (_vmap_for_bhqkv), even with attn_implementation="eager". This appears to be a frontend tracing limitation rather than a model issue. The simplest fix is to bypass the vmap path during tracing/export by falling back to legacy masking :

# transformers/masking_utils.py
if torch.jit.is_tracing() or torch._dynamo.is_compiling():
    return legacy_mask_function(...)

with this two changes , optimum-cli export and manual ov.convert_model both work without user-side workarounds.

[rutiktambe]

Crash happens there because ov.convert_model traces into a torch.vmap-based masking path in Transformers, which is not supported by the OpenVINO PyTorch frontend, leading to an internal unordered_map::at failure during graph decoding.
According to me best way to fix it is avoiding the transformers while tracing , correct fix is to bypass torch.vmap masking during tracing/export and fall back to a legacy implementation; OpenVINO currently cannot trace functorch graphs.

[Wan-Intel]

We are checking on this issue, and we will provide an update soon.

[Wan-Intel]

I've ran your code snippet and I'm also facing the same issue while converting Qwen/Qwen3-VL-Embedding-2B to OpenVINO™ Intermediate Representation (IR) format:
RuntimeError: Couldn't get TorchScript module by tracing.

On the other hand, I also ran the following command
optimum-cli export openvino --model Qwen/Qwen3-VL-Embedding-2B ov_Qwen3-VL-Embedding-2B

but encountered the following error:
ValueError: Asked to export a qwen3_vl model for the task feature-extraction (auto-detected), but the Optimum OpenVINO exporter only supports the tasks image-text-to-text for qwen3_vl.

Please open an issue at https://github.com/huggingface/optimum-intel/issues if you would like the task feature-extraction to be supported in the OpenVINO export for qwen3_vl.

I'll escalate this issue to relevant team, and we will provide an update soon.

[jinwater88]

@rutiktambe #3093 (comment)