I have created an application to extract text from image using AI

-

 In this article, I am going to demonstrate how text is getting extracted from image file using Gemini AI model. Also, will explain how the application is developed with the help of chat GPT and some custom code.

 


 

Very first i gone to chatGPT application and added below prompts so that my application structure i can get the application code to start with. Following is the screenshot of the prompt i have written and answer given by chatGPT.

chatgpt prompt 1

chatgpt prompt 2

 

 I have copied the code given by chat gpt and modified as per my requirement that means i got 70 to 80 percentage code to develop the application.

Following are the Prerequisites required for development of this application :

  1. Visual studio code
  2. Python
  3. Api Key of Gemini can be obtain from https://aistudio.google.com  

Open visual studio code and create the file with name "imagetextextractapp.py". Now in visual studio code and go to terminal menu and click on New terminal link it will open new terminal. In terminal enter below command to install the Google generative AI library, pillow and steamlit library in your machine.

 

 pip install streamlit pillow google-generativeai

 

following is the complete code which i have used after modification. copy below code to  "imagetextextractapp.py" file.


 import streamlit as st  
 from PIL import Image  
 import io  
 import os  
 import google.generativeai as genai  
 from IPython.display import Markdown  
   
 # Placeholder function for Gemini LLM interaction  
 def query_gemini_llm(image_bytes, question):  
   os.environ['GOOGLE_API_KEY']="your api key"  
   genai.configure(api_key=os.environ['GOOGLE_API_KEY'])  
   
   # set the vision model 
   vision_Mod=genai.GenerativeModel('gemini-pro-vision')  
   
   response=vision_Mod.generate_content([question,image_bytes])  
   
     
      
   return Markdown(response.text).data  
   
 # Streamlit app  
 def main():  
   st.title("Extract Text from Image with Gemini LLM")  
   
   # Image upload  
   uploaded_file = st.file_uploader("Upload an image", type=["jpg", "jpeg", "png"])  
   
   if uploaded_file is not None:  
     # Display the uploaded image  
     image = Image.open(uploaded_file)  
     st.image(image, caption="Uploaded Image", use_column_width=True)   
   
     # Text input for the question  
     question = st.text_area("Enter your question related to the image","Extract the text data in json format of given image of invoice which having Item data which should extract separately with accuracy")  
   
     # Button to submit the query  
     if st.button("Submit"):  
       if question:  
         # Query the Gemini LLM model  
         response = query_gemini_llm(image, question)  
         st.write(response)  
       else:  
         st.write("Please enter a question.")  
   
 if __name__ == "__main__":  
   main()


 In above code i have used gemini's vision mode "gemini-pro-vision" to get the information of image in textual format. In "query_gemini_llm" method takes image bytes and prompt and returns the formatted text. I have called "query_gemini_llm" method called in streamlit button click event and sent image bytes of uploaded image by user. thereafter i have print on screen using write method of streamlit library.

Now run the below command to test the application.

 python -m streamlit run imagetextextractapp.py

following is the output of program which i have tested.

output image 1
output image 2

output image 3

You can see in above output of the image extraction application is given correctly by the vision model of gemini which can be used for data entry in the other applications which will be common use case of the application. In this user can change the prompt in the text area field and fine tune the output as per user requirement.

Thank you for reading the article.


Comments

Post a Comment

Popular posts from this blog

Implement Logging in CSV file using Nlog in .net core MVC application- part 2

-
Image
In previous article " Implement Nlog in .Net core MVC application part 1 "  I have demonstrated how we can implement basic setup of NLog in .core mvc application. In previous article we have logged the exceptions in flat file system. In this article I will show you how you can write the exception and custom logs in csv file. Writing logs in csv file can enable us better readability of logs in tabular form.       Lets start the implementation of it, I going to use the same example project used in previous demonstration. Modify the Nlog.config file as follows, added new target and rule for logging logs in csv file. <?xml version="1.0" encoding="utf-8" ?> <nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd" autoReload="true" throwExceptions="fal
Read more

Implement Nlog in .Net core MVC application part 1

-
Image
Logging is a very important part of the enterprise application/web application. Setting up the logging correctly not only beneficial for long term maintenance of the application but also it is useful tool to find out the RCA in case of mission critical bug arises. Therefor we will going to discuss the various logging technique implementation using Nlog library package. In this article we will going to use .net core application for demonstration but you can implement it in .net framework application also. So lets start the implementation of Nlog in .net core MVC application. Open visual studio 2019/17 and create the new project as shown below.     Select Asp.net core Web App (Model-View-Controller) and provide the project name in project name text box.   Now on next screen click on create button this will create MVC app with .net core Now go to solution explorer and right click on project and click on Manage Nuget Package. Now in Nuget package manager install below listed Nlog packages.
Read more

Angular User Session Timeout example step by step

-
Image
This functionality let the user know that their session is about to expire and that they would be logged out if they didn’t take action. I have written this functionality in Angular for my latest application.  If you are new to Angular 6 then read my article about Setting up Angular 6 step by step before jump to this article to get handy in Angular 6. Dealing with timers in Angular can be significantly different when using React components and subscriptions. The basic premises remains the same, though. We’ll have a service with a timer that will provide a Subject to which consumers of the service can subscribe. A React Subject provides an easy mechanism to trigger a “next” subscription to alert consumers that the timer has expired.   import { Injectable } from '@angular/core'; import { Observable, Subject, Subscription, BehaviorSubject} from 'rxjs/Rx'; @Injectable() export class IdleTimeoutService { private _count: number = 0;
Read more

Restore the lost focus of Auto post back controls in asp.net update Panel control

-
Problem: We put  autopostback controls like textbox, DropDownList controls inside in update panel to reducing the flickering of the Page.but when user uses tab for entering the details  the focus of the control losts. Web site users who prefer to use keyboard need to use mouse to activate appropriate input box or press TAB multiple times. Solution: Step 1: Save following in js file give any name to it (like MyUpdatePanelFocus.js ) var lastFocusedControlId = ""; function focusHandler(e) { document.activeElement = e.originalTarget; } function appInit() { if (typeof (window.addEventListener) !== "undefined") { window.addEventListener("focus", focusHandler, true); } Sys.WebForms.PageRequestManager.getInstance().add_pageLoading(pageLoadingHandler); Sys.WebForms.PageRequestManager.getInstance().add_pageLoaded(pageLoadedHandler); } function pageLoadingHandler(sender, args) { lastFocusedControlId = typeof (document.acti
Read more

Disable backspace key using Jquery

-
Disable backspace key in HTML form using Jquery in simple steps. While user enters data into application if user accidentally  press backspace button then it will redirect to previous page and all entered data would be lost. to eliminate this event we can disable the backspace key using Jquery. Following is the JQuery script tag you can add to the web page $(document).keydown(function (e) { var nodeName = e.target.nodeName.toLowerCase(); if (e.which === 8) { if ((nodeName === 'input' && e.target.type === 'text') || nodeName === 'textarea') { // do nothing } else { e.preventDefault(); } } }); The above code can be used seamlessly with any web technology you are using. Hope this will help you in your web application. Happy coding:)
Read more