Azure OpenAI Service Security Review

In this blog, we have identified possible threats (potential causes of incidents) when using Azure OpenAI Service, and summarized what countermeasures can be taken against those threats. We hope that this will be of help to those who want to use AI services such as ChatGPT safely.

Executive Summary

After organizing the security concerns in the use of Azure OpenAI Service based on our hypotheses, we considered what countermeasures could be taken.
Since Azure OpenAI Service can use the security services provided by Azure without modification, we were able to confirm that risk mitigation measures can be taken from various angles.

What is Azure OpenAI?

Azure OpenAI is a generative AI service created through a partnership between OpenAI and Microsoft. It can use an AI model equivalent to ChatGPT, which has attracted considerable attention recently, but has the following features.

### Feature 1: Runs on Microsoft Azure and can use the same security mechanisms previously provided by Microsoft Azure
Azure OpenAI is provided as a part of "Cognitive Service", a set of AI services previously provided by Microsoft on Azure, and is hosted on Azure. No data transfer occurs. The security point is that the security mechanism provided by Microsoft Azure can be used without modification. This blog focuses on this point.

### Feature 2: GUI (Open AI Studio) enables AI model tuning and tuning data management
The "Open AI Studio" allows users to deploy AI models such as GPT-3 and Codex on the GUI. Open AI Studio also has a screen called a playground, where you can enter natural language prompts to see model responses and tune parameters.

### Feature #3: SLA is defined at 99.9%.
SLAs are defined because the service is provided as an Azure service. At this time, there is no API response time service level agreement (SLA) defined, but the overall SLA for Azure OpenAI Service is 99.9%, the same as other Azure Cognitive Services.

## What are some of the threats when using Azure OpenAI?
Before discussing the security of Azure OpenAI, the premise is that security incidents occur when "threats" and "vulnerabilities" match. Therefore, in this section, we will identify what threats exist from the user's perspective, and then we will show how many measures can be taken to reduce vulnerabilities to those threats.

When it comes to full-scale threat modeling, various approaches such as data flow diagrams and sequence diagrams are taken, but this time we will take a simplified approach based on the above image diagram. The flow of construction and processing (circled in green) in the above diagram is as follows.

1. The user selects a basic model and deploys Azure OpenAI Service to their tenant.
2. (Optional) The user prepares data for fine-tuning and tunes the AI model for their own use; the fine-tuning data and the fine-tuned model are stored in Azure Strage.
3. The user inputs questions to the AI model, which can be done via API or GUI if the user has Open AI Studio privileges.
4. The AI model predicts the next flow based on the input data and generates outputs.
5. The output is returned to the user via GUI or API.
6. The input and output data are stored in a Microsoft database for 30 days.
7. If Azure OpenAI detects that a user may be using Azure OpenAI for prohibited purposes, a Microsoft Specialized Unit will be alerted.
8. Microsoft's specialized unit will check the input and output data, etc., and take necessary action if it is determined that the data is being used for prohibited purposes.

## Ⓐ.　Threats related to input data
### inputs inappropriate data
#### Threat Description

The first concern is that "inappropriate data may be input. For example, this is the case of inputting information such as "personal information that is not supposed to be shared outside the company based on a contract with a customer" or "information on a new product before it is released. The following measures can be considered for this case.

#### Measures: Data Classification/Labeling and User Education

The first measure is not a systemic measure, but rather a measure of data classification/labeling and user education. Without company-specific indicators for what constitutes "inappropriate" data, it is impossible to start this discussion in the first place. Therefore, we think it is desirable to classify the information handled by companies and sort out "information that can be shared to Azure OpenAI" and "information that is not". The story starts there.

#### Measures: Implement and audit a history function
When providing Azure OpenAI to users, it is conceivable to implement a history function and use that history to audit, or implement your own filtering. This is an area of system development, and there are many different implementation methods, but here are a few reference examples.
- Upload historical data to Log analytics. Use that data to audit with Sentinel.
- Upload historical data to Azure Cosmos DB (NoSQL).
- Enable users to use Azure OpenAI services via Teams chat using Microsoft Power Virtual Agents, and audit Teams chat history with Communication Compliance, etc.

### Malicious use and generation of false information (deep fakes)
#### Threat Description
Generated AI, including OpenAI, can also be used in morally objectionable cases. Examples include generating phishing attacks and developing malware. This could threaten the information of targeted companies or individuals. Generative AI also has the ability to generate deep fakes (video or audio that falsifies a person's face or voice). This can result in damage to a person's reputation and invasion of privacy.

#### Measures: Microsoft's detection system
Azure OpenAI Service stores input and output data for up to 30 days, and Microsoft uses the stored data to monitor for any use of the service that may violate the Code of Ethics. If a policy violation is identified, Microsoft will take action to correct the usage, suspend access to Azure OpenAI resources, or terminate usage, so that **some degree of** protection against malicious use and generation of false information (deep fakes) is in place by default. Measures:

#### Measures: In-house education

Although Microsoft has taken countermeasures, it is advisable to educate your own users when you provide AI to your own users. Measures such as setting up and informing users in advance of the expected uses and prohibitions are considered.

## Ⓑ. Threats to output data
### Violation of public scope of output data

#### Threat Description

Data generated by Azure OpenAI Service (output data) may contain the contents of input data and data used for fine-tuning. This means that people who are not allowed to view input data and fine-tuning data must not view output data either. I think this is a blind spot, especially for fine-tuning data.

#### Measures: Designing output data so that it does not violate the scope of publication of output data

There is no way to solve this problem except to design the output data in such a way that it does not violate the disclosure scope of the output data. It is also advisable to design systems and guidelines with sufficient attention to secondary use of output data.

**Caution**.

If you are concerned that the usage of output data may violate the terms of Azure OpenAI Service, please refer to Microsoft's Azure OpenAI Use Cases, Cautions, and Best Practices.

### Copyright Issues

#### Threat Description

There are multiple aspects of copyright issues related to generative AI, but this article focuses on infringement of copyrights of others. If the products from generative AI are identical or similar to existing copyrighted works, the use (reproduction, distribution, etc.) of such products may constitute copyright infringement [9], and companies need to take this into consideration when using them.

#### Measures:.

We believe that society has not yet caught up with this copyright issue, as not many cases have been reported. As a practical measure, when using Azure OpenAI Service, educate users not to enter the names of existing works, authors, or works in the prompts, and when distributing or publishing output, check that the output does not resemble existing works.

**Note**

Regarding copyrights, we have asked "Is it acceptable to input copyrighted content into a generated AI?" and "Does learning with copyrighted content constitute copyright infringement? and "Does learning with copyrighted content constitute copyright infringement? We recommend that you consult with your legal department while referring to the opinions of law firms and other organizations.

## Ⓒ. API Threats

#### Threat Description

There is a threat of malicious third parties exploiting the APIs, resulting in the exploitation of information or AI models. Microsoft's existing security measures can be taken against these threats, and some are introduced here.

#### Measures: Auditing with Sentinel (Log Analitics)

Logs (diagnostic information) related to Azure OpenAI Service can be sent to Log Analitics and analyzed by Sentinel and others. For example, the IP address of the client side when the API is executed can be obtained, so it is possible to use them to identify the problem.

#### Measures: API Execution Restrictions

The above mentioned measures are auditing, but another option is to restrict the IP addresses or networks that can be accessed, such as using IP addresses, virtual networks, or private endpoints.

**Note:**

Even if you restrict access by IP address or private endpoint, some functions of "Open AI Studio" can still be used.

**Others:**

In addition to the above measures, Azure Key Vault, Azure API Management or Micrsoft Defendr for API may also be used.

## Ⓓ. Threats related to own tenants and GUI
### Azure own tenant is compromised

#### Threat Description

Azure OpenAI Service, like other cloud services, is at risk of tenant compromise. Specifically, a malicious party (external or internal) misuses its own Azure tenant and compromises data and resources. Cloud security is based on a "shared responsibility model," in which users and cloud service providers each define their own areas of responsibility and take their own countermeasures.

#### Measures: Configure Azure security settings

Azure has many security features by default, and by faithfully implementing them, it is possible to significantly reduce risk. There are too many to list them all here, but here are a few that we have picked up from Azure security documentation and other sources. For comprehensive information, please refer to Microsoft's documentation.

- Multi-factor authentication (MFA).
- Conditional access to prevent access except for terminals in an authorized state.
- Separate subscriptions and resource groups for production and verification to reduce the risk of mishandling.
- Use RBAC (role-based access control) to prevent unwanted users from accessing Azure OpenAI Service resources.

### **GUI screens provided for users are compromised**.

#### Threat Description

If ChatGPT is to be used for general users, it is necessary to provide a GUI screen, etc., but providing such a screen naturally makes it an attack surface as well.

#### Measures:.

I think it is difficult to introduce general measures here, because the solution will depend on the form of the GUI screen to be provided to the user. However, I personally think it is very eco-friendly to use Microsoft Power Virtual Agents as mentioned above to make Teams chat GUI, because the security measures that have been taken in Teams can be used as is.

## Ⓔ. Threats related to data stored on Azure
### User data stored on Azure will be compromised

#### Threat Description

D introduced "threats related to own tenants and GUI", but the threats introduced here are related to the scope of responsibility on Microsoft's part: threats related to the fact that the infrastructure and physical parts of Azure will be compromised.

#### Measures: the complemented data is encrypted

Azure OpenAI is part of Azure Cognitive Services, and Cognitive Services data is encrypted by default with data using a FIPS 140-2 compliant 256-bit AES cipher. The encryption key in this case is managed by Microsoft.

#### Measures: The encryption key used can also be the company's own encryption key.

In the above case, the encryption key is managed by Microsoft, but users can also bring their own key (BYOK). By using this, the key cannot be read by Microsoft, and even if Microsoft is compromised, the data will be protected in encrypted form as long as the encryption key has not been leaked.

#### Measures: Other measures Microsoft has in place to protect its customers' data

Microsoft has implemented world-class security measures to protect customer data. They comply with security certifications and standards around the world, and have invested tremendous amounts of money in the areas of security and privacy.

### Foreign laws and regulations apply to data stored on Azure

#### Threat Description

Next to the use of cloud services, there is a possibility that user data will be stored on overseas servers and that overseas laws and regulations will be applied. The application of foreign laws and regulations is not in itself a bad thing, but depending on the content of the laws and regulations, there is a concern that something unthinkable under Japanese laws may occur.

#### Measures:.

Azure has multiple regulations, and the regulations applicable to each company and each specific case are different. Therefore, in practice, it is advisable to consult with your company's legal department, etc., while taking into consideration the facts below and the actual agreement you have with Microsoft.

- Currently, Azure OpenAI is only available in three regions: East US (Eastern US), South Central US (South Central US), and West Europe (Western Europe).
- Azure services, including Azure OpenAI Service, are governed by the Microsoft Customer Agreement and Online Subscription Agreement, which, if the "Country" is Japan, will be governed by the laws of Japan in the Tokyo District Court of Japan.
    - Microsoft Customer Agreement : <a href="https://www.microsoft.com/licensing/docs/customeragreement" target="_blank" rel="nofollow">https://www.microsoft.com/licensing/docs/customeragreement</a>
    - Microsoft Online Subscription Agreement : <a href="https://azure.microsoft.com/ja-jp/support/legal/subscription-agreement/" target="_blank" rel="nofollow">https://azure.microsoft.com/ja-jp/support/legal/subscription-agreement/</a>

## Ⓕ. Threats used to learn the basic model
#### Threat Description

The term "basic model learning" may not ring a bell, but the concern is that data input/output by users (user data) could be used by Microsoft or OpenAI to learn their basic models (which may be used by other customers). This is a concern.

#### Measures:. 
First, Azure OpenAI Service states that user data will not be sent to OpenAI. Microsoft also states in their support that user data will not be used to train models for Azure OpenAI Service. Therefore, unless the Azure OpenAI Service terms and conditions change, Azure OpenAI Service user data will not be used to train the basic model.

## Ⓖ. Threats related to access by Microsoft employees.
#### Threat Description

As mentioned earlier, Microsoft stores input and output data for up to 30 days and monitors possible service use that violates product terms and conditions. While this is a good aspect of the "malicious use and generation of false information (deep fakes)" countermeasure, it means that Microsoft reads the input and output data, which could lead to a violation of its own security policies. Also, although I believe Microsoft carefully selects the personnel who perform that task, there is a risk of unauthorized use if those personnel have malicious intent. (I know it's unlikely, but...)

#### Measures:. 
Users can request to opt-out of filtering and abuse monitoring. If approved in compliance with all requirements, input and output data will no longer be stored and cannot be viewed by Microsoft employees. If you cannot allow Microsoft employees to view the data due to a conflict with your company's security policy or other reasons, you may want to consider this option.

## Conclution
Personally, I think Azure OpenAI is more recommended than ChatGPT for security-conscious use of generative AI. I think ChatGPT is mainly for personal use, but I would like to try ChatGPT Business, a service for business use, when it comes out.

### References
- Azure OpenAI Service [https://azure.microsoft.com/ja-jp/products/cognitive-services/openai-service](https://azure.microsoft.com/ja-jp/products/cognitive-services/openai-service)
- Transparency Note for Azure OpenAI Service [https://learn.microsoft.com/ja-jp/legal/cognitive-services/openai/transparency-note?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext&tabs=text](https://learn.microsoft.com/ja-jp/legal/cognitive-services/openai/transparency-note?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext&tabs=text)
- Thread Modeling | OWASP（Open Web Application Security Project） [https://owasp.org/www-community/Application_Threat_Modeling](https://owasp.org/www-community/Application_Threat_Modeling)
- Thread Modeling | Microsoft [https://docs.microsoft.com/en-us/azure/security/develop/threat-modeling-tool](https://docs.microsoft.com/en-us/azure/security/develop/threat-modeling-tool)
-  [Data, privacy, and security for Azure OpenAI Service – Azure Cognitive Services | Microsoft Learn](https://learn.microsoft.com/ja-jp/legal/cognitive-services/openai/data-privacy?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext#how-is-data-retained-and-what-customer-controls-are-available)
- Advanced Encryption Standard [https://en.wikipedia.org/wiki/Advanced_Encryption_Standard](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)
- Azure OpenAI Service encryption of data at rest [https://learn.microsoft.com/en-us/azure/cognitive-services/openai/encrypt-data-at-rest](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/encrypt-data-at-rest)

GUCHY NOTES

Search This Blog