Ask Hashtag! OpenAI in the Power Platform

by Kylie Kiser, posted in AI Builder, Citizen Developer, MSDyn365, Power Apps, Power Automate, Power Platform

OpenAI is now available in the Power Platform! The new OpenAI model is part of AI Builder and currently in Private Preview!

Anyone can sign up to join the Private Preview. Note that you will need a sandbox environment and the environment/tenant details to join the Private Preview. Once you are added, you can see the new connector within AI Builder then add to Power Apps or Power Automate.

OpenAI AI Builder model
OpenAI Service will be available when creating a new model in AI Builder in the maker portal
Test out OpenAI Connector in AI Builder
Once the OpenAI Service is visible in AI Builder you can test it out and see samples

Ask Hashtag

We are going to use the OpenAI Service in AI Builder to build this app to talk to Hashtag!

Ask Hashtag App featuring a text box, generate response button, response box, then selector for language to speak, button to generate audio then Audio play button.
Quick preview of the Ask Hashtag App. Watch the video to see full functionality.

Check out this video to see the full functionality:

Quick Demo of OpenAI in the Power Platform and the Ask Hashtag! App

How to

Now, let’s talk about how this app was built. We do not have time to walk through each piece step-by-step but we can cover the highlights. You can also check out the Learn content on using OpenAI in Power Apps.

Prerequisites: Environment with an AI Builder Trial and the OpenAI Preview enabled.

Within your Canvas Power App, you will add new data and Select the OpenAI model under AI Builder. This will allow you to reference the model in Power Fx.

Add AI Builder Model
Add Create Text with GPT Azure OpenAI Service model as a data source in your Power App

Now you can add a text box to input data and another box to show the response. I created a button to run the model and get the response. Here is a sample formula:

Set(Response, 'Create text with GPT'.Predict(TextInput1.Text))

This will set a new Variable Response to be the GPT Response to the prompt written in TextInput1. Then you can display this.

Next up, we want to synthesize the response so that we can hear Hashtag talk! To do this we are going to use the IBM Watson Text to Speech connector. We could do this with Azure Cognitive Services as well but there was not an existing connector.

To use the IBM Watson Text to Speech connector you will need to create an IBM Cloud Account and enable the free speech synthesizer plan. When you do this you will get an API Key and URL that will be needed when using this connector.

I put this logic in Power Automate. This means that our Power App will call a Flow to generate the audio. Then the audio file is returned and played from the app. We can also use the AI Builder Translation service to translate text in this Flow before we create the audio file.

Here are some screenshots of the Flow in Power Automate:

Ask Hashtag Get Audio Flow Design
The Flow gets data from Power Apps, creates a variable, uses a switch statement to synthesize the audio, then sends the variable back to Power Apps
Ask Hashtag Get Audio Flow - Synthesize English
The first case for English will just Synthesize the audio using the IBM Text to Speech connector and set the variable.
Ask Hashtag Get Audio Flow - Translate and Synthesize
Case 2 and 3 will translate based on the language selected then Synthesize and set the variable. In this case, we only used 3 languages. If we wanted to add additional languages we could add additional cases.

The code to generate the Audio looks like this: (onSelect for the “Generate Audio” Button)

Set(AudioResponse, 'HashtagSpeaks-GetAudio'.Run(Dropdown1.Selected.Value, Response.Text))

Then the code on the Audio Play bar looks like this: (onSelect for the Audio control)

AudioResponse.audio

Next Steps

This is just a quick, fun use case for trying out OpenAI, Independent Connectors, and other AI Builder models! I hope it helps you get started with your Artificial Intelligence journey in the Power Platform. Let me know what other content you would like to see!

One thought on “Ask Hashtag! OpenAI in the Power Platform”

  1. Pingback: Ask Hashtag! OpenAI in the Power Platform | 365 Community

Leave a Reply