How Azure OpenAI Enables Data Privacy for Generative AI Models

Unknowing users shared sensitive corporate data and confidential source code with open-sources models like GPT, which rely on user inputs and queries to upgrade the model performance.

Behind-the-scenes data collection by various Gen AI models is problematic for several reasons:

• Lack of explicit consent on using certain personally identifiable information (PII) for model training purposes. Even if the user submitted their data willingly, its usage can be a breach of contextual integrity — a legal framework that emphasizes the preservation of information flow norms within specific social contexts (i.e., not being revealed outside of the context in which it has been produced).
• No traceability. Users cannot request which data has been collected about them, where it is stored, and how it can be removed. These requirements, however, are mandatory under global privacy laws like GDPR, for example. Regulators in Italy, Germany, and France are already investigating whether OpenAI has the right to use people’s personal information for model training.
• Intellectual property (IP) breaches. Artists and media companies have already raised alarms that Gen AI models used their work for training without obtaining their respective permission. Submissions of source code (which have become frequent) to ChatGPT can also result in disclosures of commercially sensitive information.

Lastly, like any other open-source tool, ChatGPT has some inherent security vulnerabilities, which can result in data breaches. One known prompt injection vulnerability can force new prompts into your ChatGPT query without your knowledge or permission to steal data.

Generally, 89% of tech leaders are concerned about the impact of generative AI on cybersecurity. However, the rightful concerns do not mean that ChatGPT for business is fully off-limits.

In early 2023, Microsoft released Azure OpenAI service, offering businesses the ability to deploy foundation models like Codex, DALL-E, and ChatGPT on Azure infrastructure.

How Does Azure OpenAI Improve the Data Privacy of ChatGPT?

ChatGPT collects and hosts all user data on its own servers — and that is problematic. Azure OpenAI Service provides users with access to the same underlying foundation model (GPT) plus the ability to host it on a managed Azure instance. Similar to other types of cloud data storage, Azure handles infrastructure management and optimization without having direct access to the stored data (or allowing its usage by any third parties). Effectively, you gain access to a private version of ChatGPT for business usage.

Business Breakfast

Learn more

Microsoft Cloud and AI in Action

Apart from controlling the generated data, Azure OpenAI also lets you supply extra data for training the GPT model. For example, authorize access to the corporate Code of Conduct document to train the model on the corporate principles.

The above scenario is possible thanks to Retrieval Augmented Generation (RAG) — a framework for incorporating external knowledge to pre-trained large language models (LLMs) to augment its intrinsic knowledge with fresher, domain-relevant information.

​​By combining Azure OpenAI service and Azure cognitive search for data indexing and retrieval, you can operationalize knowledge reserves within your organization without risking any external exposure. The deployed model does not exchange data with OpenAI and you remain in full control over the information it accesses. Effectively, you can run a private Azure ChatGPT version, training specifically for one team, business function, or internal corporate usage. All authorized users can then interact with the model via Microsoft Teams.

Apart from training OpenAI service on text documents, you can also configure it to process numerical data from databases, spreadsheets, or connected analytical models (e.g., Power BI). Doing so enables your teams to query analytics using free text inputs such as “How many sales did we make for product X in Q1 2024?” to quickly interpret available data without calling upon a business analytics specialist.

Benefits of Using Azure OpenAI for Business Use Cases

• Corporate data privacy: Azure OpenAI security and data privacy are unquestionable. No sensitive information gets shared with OpenAI or Microsoft. All the data, processed by the model, remains within the corporate perimeter. Respectively, there is no extra exposure to compliance or regulatory risks. Azure OpenAI offers full VPN and private endpoint support for your data.
• Higher model accuracy: You can augment the general GPT model “knowledge” with extra data from any number of connected sources — from your corporate wiki to the customer support platform or even a connected SAP HANA database. Thanks to RAG, the model can retrieve corporate knowledge and provide more contextually and domain-relevant responses to users.
• Flexible controls: Maintain full control over model access permissions and usage volumes. You retain full control over chat session states — limit access to certain information to high-privileged users only to avoid accidental internal disclosures or customize model behavior for different groups of users.
• Simple integration. All models are packaged as a well-documented API and can be configured for a wider range of tasks — from document summarization to machine translation and more. Azure OpenAI service also has a number of native connectors to other Azure services and tools.
• Extensive documentation and support. Microsoft provides full technical documentation and reference architectures for deployment, covering various usage scenarios.
• Fast time-to-market. Because the distributed models are already pre-trained, the implementation process can take as little as four weeks with an experienced product development team. 

Top 3 Scenarios for Deploying Azure OpenAI for Corporate Usage

Over 65% of business leaders are ambivalent or dissatisfied with the company’s progress on AI and GenAI implementation. Indeed, developing a custom machine learning model or training an LLM in-house is a labor- and resource-intensive project.

Similar to how low-code platforms like Power Apps can automate a wide range of standard business workflows, Azure OpenAI service can be used to streamline a number of knowledge tasks to augment workforce productivity.

1. Working with Internal Documentation

With Azure OpenAI, you can run GPT-35-Turbo and GPT-4 models against your own data without training or fine-tuning them. Effectively, you supply the model with extra data for analysis and receive optimized responses (e.g., a text summary of all applicable organizational policies for new hires).

Instead of manually browsing web pages or scanning the corporate file storage systems, users can shoot a quick message to an AI model to look things up for them. With properly implemented RAG, Azure OpenAI models produce highly accurate results with few hallucinations and inconsistencies.

Sample Scenario: A conversational assistant for the HR department, providing data on the organizational structure, current roles, compensation plans, performance review schedules, role descriptions, and pretty much any other information your team needs for workforce planning and management.

2. Automatic Report Generation

Another clever way of using Azure OpenAI is report creation. Whether you need the latest data on customer churn or a tally of the marketing budget, the foundation model can handle the tasks within seconds.

By connecting Azure OpenAI service with different data sources — Power BI, Dynamics 365, or an ERP system — your teams can quickly get the data they need without doing any coding or running formulas. The GPT model can crunch the numbers and present the information in a more accessible manner. 

Example: Infopulse team recently created a data analytics chatbot for a major pharmaceutical company. It can access data from a connected Power BI instance and provides text interpretations, visual diagrams, or custom Excel reports based on the user query. Apart from making data analytics readily available to any number of users, the company also substantially saves on Power BI licenses and extra labor hours, associated with dashboard setup and data querying.

3. Internal Knowledge Management

Azure OpenAI can be easily integrated with a digital workplace platform (e.g., made accessible via Microsoft Teams). Instead of pinning colleagues or browsing the endless string of chat messages, an employee can shoot a quick question to the GPT and get a relevant response. BCG estimates that Generative AI assistants can increase the productivity of customer service professionals by 30% to 50%. Similar results can be achieved for other functions — sales, marketing, finance, operations, IT services, and cybersecurity among others.

Pro tip: To avoid sensitive information disclosures, schedule a data discovery session with Microsoft Purview first. The service lets you classify and label all the available data from connected data sources as “public”, “internal”, “confidential”, or “restricted”. Based on these labels, you can configure which information classes generative AI model(s) can access.

Example: A corporate virtual assistant for telecom, trained to provide answers on all product specifications or service offerings, latest pricing information, volume discount structures, sales level agreements (SLAs), and other terms for new and existing customers.

Conclusion

With Azure OpenAI Service, companies get the best of both worlds: Access to robust, pre-trained generative AI models and unquestionable data privacy. Your organization retains full control over data, application customization, and infrastructure. A private Microsoft OpenAI instance with a GPT model is also more cost-effective than having an equivalent number of paid business subscriptions to ChatGPT.

Infopulse team has already helped a number of global businesses deploy private generative AI assistants for a wide range of use cases — from corporate knowledge management to data analytics. Contact us for a personalized demo!