AI Meal Analyzer - Upload and Display Images with Flask

Hi, welcome back to this Python for AI apps course with Flask and JavaScript. My name is Brian McLean. We have completed project one, which was the AI chat assistant.

In project two, we have a new project folder. So you'll need to set up your virtual environment, which you can refer to the very beginnings of the book, lesson double zero, on how to do that. Let's take a look at the setup environment we're in.

And we're in VS code still, but we're in a new folder environment. And then let's run the final file so you can see what we're going to be making here. Okay, so we're in this new folder.

It's part two, it doesn't have any of the stuff from the AI chat assistant. We've got a file called done, py done, html done, and there's a js done. So of course they all work together.

What we want to do is run the done py file, the done server, and then go look in the browser what we have. So I've already run it, so I just hit the up arrow. Make sure you're in venv.

Again, you have to go and look at lesson double zero on how to set up your venv environment. You'll look over here, you'll see I have an env and a git ignore. You can ignore those for right now.

And we're using the same API key as we had last time. And if you go into the oy py file, the 01 server meal analyzer 01 py file, the only thing in there is the API key, same key as before. Of course your keys aren't going to work.

You'll have to have your own API key. This key won't work, right? So I'm going to run the server and then check in the browser. We're going to go to visit that url in the browser, our little local home server here.

And we've got this AI meal analyzer. It says upload a meal image to AI to get preparation instructions, calorie counts, and macronutrients, carbs, fat, protein. Check boxes for vitamin and mineral profiles.

Let's not, there's little check boxes. Let's just not do the check boxes. Browse.

Your new project folder has some meal images. Certainly feel free to add as many other pictures as you like of food. Hit open.

And in this first lesson, we're just going to get into how to use HTML and JavaScript to do what you see right here, which is load up an image. But this is the final file. So it actually works.

It's already connected to the AI and everything. Let's see. I just want you to see what we're building towards.

We hit analyze. It'd be great if we had a little spinner in here. That was pretty quick.

Stuffed avocado quinoa delight. That looks about right. A vibrant and nutritious ensemble featuring ripe avocados filled with fluffy quinoa, cherry, tomatoes, pumpkin seeds.

Yeah. Wow. It found the pumpkin seed.

There is a couple of pumpkin seeds in there and fresh herbs. The meal offers a delightful balance of texture and flavor. Cooking instructions, total calories, itemized calories by ingredient, protein, fat, and carbs.

That's pretty good. Now let's click minerals and vitamins and load up the same, that exact same quinoa avocado boat, as they sometimes call it. Analyze.

This will take longer. The prompt is much longer. It's making a full nutritional analysis.

All the various vitamins and minerals. There you go. So it's working great.

Let's go back to the code and we've got the book cooking here. We're in lesson 12 of the book, part two, lesson 12, making a file chooser. So part two is building an image analyzer.

Let's just fix that. And now we are lesson 12, making a file chooser in HTML and JavaScript. So there would be no open AI yet, right? Just like with the, that AI chat assistant, we built over many lessons to that.

We're going to start just with trying to get that image to load up, to build a browse to an image, not submit the image for analysis. There's no server side. It's just JavaScript browsing to your local system.

So from meal analyzer 00, let's do a save as and call the new file meal analyzer 01. All right. I have it at one.

Let's delete that meal. Okay. Meal analyzer 00.

Let's look at what that is. It's just this. From these humble beginnings, we will build our awesome AI meal analyzer app.

And there's the body. Not much going on. We need to add a few more HTML elements, namely an input of type file for choosing an image to submit for AI analysis.

It could be any image, but our prompt is instructing the AI to be on the lookout for food. A button for calling a JS function, which will send the image data to Python flask, flask server, an image for displaying the uploaded chosen meal image, an IMG tag. Right.

It's not enough. I mean, you wouldn't have to do that. You could just browse to the picture and not show the user the picture.

They would have to know what it looks like, but it's a much nicer UI if the user gets to see the picture uploaded into the browser. And that's all done with just HTML and JavaScript. So above the H3, the meal name, add an input element of type file.

So you know there's type text and type checkbox and type number, all these different types of input. Type file produces this little choose file button that allows you to browse your system. The accept attribute is for specifying the file types which are acceptable for upload, and you just list them out as image, in this case, slash file format.

File type will allow PNGs, JPEG with an E, without an E, that newfangled WebP format, and GIF. So just images, but five different kinds of images will be acceptable for upload. So let's go in and do that.

But we have to save our file. Saving meal analyzer OO as meal analyzer O1. Right.

We don't want to type in the OO file. Just leave it alone. So above the meal name H3, we're going to say input type equals file.

Accept. We're going to accept image, slash, JPG, space, image, slash, JPEG, doesn't matter the order you put these in, image, slash, PNG, image, slash, GIF, image, slash, WebP. So it's a string.

Yep, there it is. Below the input, add a button to submit the image for analysis. We'll just have a button.

It'll have one word, analyze. Button doesn't need an ID or anything. It's the only button on the page.

We'll just grab by the button in JavaScript. We'll do a query selector on the button. Import the JS file, which we need to make next.

That's going to be a one meal analyzer. So down here, we'll say script. Is there a container? Yeah, there's a container.

I'm missing the div. Closing div. Yeah, we've got the div here.

So right before the closing body tag, let's probably provide that as context. Div. Closing body.

We don't need that. We get it. It's the last thing before they close the body.

MealAnalyzer01.js to go with meal analyzer01.html. Preview the page in the browser. Sure. That's all you got.

It doesn't work yet. Click. I mean it browses, but it doesn't do anything.

It doesn't show the picture. Step two. Defining a JavaScript function for browsing to images.

Well, the file chooser HTML thing does that by itself. Let's call it loading image. Loading and displaying images.

From the JS folder, we're going to open up the file called MealAnalyzer00.js. There it is. Got nothing in it other than comments. Not just a blank.

We're not making the file from scratch like we were doing in the previous challenge, right? The previous project. It has a few comments indicating where to type code, but no actual code yet. So do a save as and call the new file MealAnalyzer01.

In MealAnalyzer01, get the elements for image upload. Those would be the ones you just typed. The input, the button, and the img tag.

And the image. Actually, we skipped. Did we skip the image? Yeah, we did.

Backing up. We need an image tag. We don't need this.

Okay. So in our HTML, we need an img tag of id uploaded img. And we're going to put an empty source here.

Empty string. The source will get set by the uploaded image. And we'll just give the image an id, even though it's right now the only image on the page.

But that's fine. Back to step two. Okay.

So in MealAnalyzer01.js, get the elements for image upload. That being the input, the button, and the img tag. Get the DOM elements for image upload.

That would be const. What did we want to call this first thing? File input. Great.

Okay. Yeah, it's an input file. It's an input of type.

Type file. Fair enough. File input.

Not a great name. You could call it something else. But it is an input, though, of type file.

Document, query selector, input. Const, send button, the analyze button. You know what? I'm just going to call this input.

And I'm just going to call this button. Input and button. You know what? I'm not learning the hard way.

Don't do that. It's too many changes to make later. Fine.

I'm not in love with these names. It's a send button, but it says analyze. It's fine.

Document, query selector, button. And then, const, uploaded image. It's going to be document, query selector, img.

So, the image that's uploaded is going to appear in this img tag, img element. We're going to have the input next call a function when clicked. The event on that is change, not click.

So, call a function which runs when user browses for file by clicking the file input. So, we're going to say file input is dot add event listener. That will be a change event, comma, display uploaded image.

We'll call it the function. We're going to have it run display uploaded image. There we go.

And we need a couple of variables in the global scope, which means they can be available anywhere in the script. And we have two different functions, one which runs on upload and the other which runs on the which handles the display of all the loaded data. So, we're going to have global variables.

One called the image source. And the other is going to be the uploaded file. That whole data, it's a big chunk of data that we upload, right? When you browse to the image, you get this whole data as a big chunk of, they call it a blob, actually.

A blob of data is the uploaded file. So, we'll call that uploaded file. Sometimes it's literally called blob, the uploaded data file.

But uploaded file is okay. But I've seen it called blob. Let uploaded file, and we're not going to give it a data type or anything, let img src, same deal.

Declare, but don't give a value. Next, we're going to define the function which uploads an image from the user's computer. And it runs when you click, like when the user clicks that choose file.

So, function display uploaded image, scroll to the end. All right. So, what are we going to do inside here? We've got a little, we've got some indented comments.

The file input object stores uploaded files as an array called files. This being our first and only file will be stored in file zero. So, we're going to save that to a variable.

That's going to be saved to the uploaded file variable. We're going to say file input equals uploaded file dot files zero. Excuse me, I got it the other way around.

All right. File input dot files zero. So, the file input, you browse to a file, it's going to store it in an array because it's set up to be able to handle more than one file.

And in that array, it'll be the first item at index zero, and we're going to save that as our uploaded file. And again, that's a blob object. We can, in fact, we're going to at the end, we're going to log the data type of this.

Next, we're going to set the image source. So, we're going to take this uploaded file blob thing, and we're going to pass it to this URL dot create object URL, and that's going to return the image, the usable image source. So, we're kind of taking this loaded binary large object, aka blob, and turning it into a useful, usable image source.

And this stuff is all described here, including what's happening in the code, what a blob is, all that. So, read all about that if you like. Of course, this is not a JS course, and that's fairly advanced JS anyway.

So, the image source is going to be set equal to this URL dot create object URL, taking the uploaded file as argument. It's going to return a usable image source. And here it describes what you would get down in the notes.

The returned URL is a string representing a blob URL with some temporary file name. It's using a temp name at that point for the file. So, the image source is going to be URL dot create object URL uploaded file.

And it's actually the image source. We're going to set the source of the image element to the image source. So, that will be uploaded image dot SRC equals image source.

That will make the picture appear in the box, in the image as in the image, right? It'll appear on the screen in the browser. So, now you can refresh the browser and click the choose file button, which is not really a button. It's an input of type file.

Browse to an image on your computer, and it should appear in the IMG tag here as the image on the page. So, this not, it's not really that much code, right? A few lines. Let's check it out.

There's your image holder. And it didn't work. Okay.

What's going on? Nothing going on. All right. Display uploaded image.

Display uploaded image. We're using the right, oh, we're not importing the JavaScript. Okay.

Script, SRC, dot dot slash static JS meal analyzer 01 dot JS file we just were typing in. That would explain why it's not working. All right.

Let's try again. There you go. And in the next lesson, the analyze button will send the data off.

There'll be a few steps, but there we go. We're off to the races. Yeah, there it is.

Okay. That completes this first lesson in the meal analyzer. Got another picture we can use instead if we want.

They're all good. All right. So, come back for more second helping coming up in the next lesson.