Protected material

Microsoft has recently introduced a new feature to its Azure AI Evaluation SDK, focusing on identifying and assessing the presence of “protected material” within the outputs generated by the language models (LLMs) supporting various applications. This can include content such as copyrighted text, like song lyrics, recipes, and articles, which the enterprise may not have the rights to use. To facilitate this evaluation, the SDK compares the LLM’s responses against an index of third-party text content stored on GitHub.

According to the lead engineer, users can now delve into the evaluation details to gain insights into how their applications typically handle these prompts and understand the associated risks more effectively.

Additionally, the enhanced SDK offers two APIs: one is designed to identify protected copyright text, while the other focuses on detecting protected code elements, such as software libraries, source code, algorithms, and other programming-related materials.