Cisco has released an open source tool called the Model Provenance Kit to help companies address security and compliance issues related to using external AI models, SecurityWeek reported on April 30.

Companies often import third-party models from AI model repositories such as HuggingFace. HuggingFace has millions of models registered.

Cisco said three problems arise when companies bring in models from sources such as HuggingFace. First, companies do not determine how the imported models are later changed. Second, they do not verify information provided by model developers, such as model provenance, vulnerabilities and training bias. Third, maintenance levels vary by developer, affecting companies that use the models.

Cisco said these issues can lead to security, compliance and legal liability risks. A company could deploy a model embedded with malware or vulnerable to tampering. It could also adopt a model with biased training data. Cisco explained: "If you cannot identify the provenance, you cannot trace the root cause when an incident occurs, and you cannot determine whether other models in the system you use were affected."

The Model Provenance Kit consists of a Python-based toolkit and a command line interface. It generates a "fingerprint" for each model by analysing signals including model metadata, tokenizer similarity and weight-level signals such as embedding structure, normalization layers, energy profile and weight comparisons.

The tool operates in two modes. The "compare" mode compares two models to find a shared lineage. The "scan" mode matches a specific model fingerprint against a fingerprint database built by Cisco to find the closest lineage.

Cisco said: "As models go through fine-tuning, distillation, merging and repackaging, tracking lineage is becoming more difficult." It said the Model Provenance Kit is "a first step" toward presenting an evidence-based method for verifying provenance.