Purpose

For our PCOMP final, we're creating a device that allows users to ‘recreate’ their memories using an AI image generator (we chose Dalle). The device consists of a control panel with knobs, buttons and sliders that will allow users to change the prompt for the image generation. We would use Arduino's serial communication to send the prompt variables to p5.js to reflect on an interface, and then use p5.js to display the image after the API returned the URL.

Figuring out the API

As the first step in our project, we needed to figure out how the OpenAI API worked. I'm working with my friend Liyan, who has background in code and sort of knows what she's doing, so she's been really helpful in figuring all of this out.

We started by connecting the OpenAI API to Google Sheets, and thought about using it to send the prompts to Dalle and return the images to P5.js. We could get it to work, but we were having a hard time connecting Sheets to p5.js. We thought about setting up a server to do the work, and scheduled office hours with the coding lab team to help us do so.

Liyan set up server locally on her computer, and we started working with p5.js locally instead of using the web editor. We thought it'd be easier to use the server as the middle point in the communication between all these moving parts

We connected the OpenAI API to the server and wrote a p5.js sketch to send the prompt back to Dalle.

Our server and the p5.js sketch.

Receiving the response from Dalle after running the code.

The sketch - so far!

The image created with the prompt!