Posted by Paris Hsu – Product Supervisor, Android Studio
At each stage of the event lifecycle, Gemini in Android Studio has change into your AI-powered companion, making it simpler to construct top quality apps. We’re excited to announce a big growth: Gemini in Android Studio now helps multimodal inputs, which helps you to connect photos on to your prompts! This unlocks a wealth of latest potentialities that enhance workforce collaboration and UI growth workflows.
You’ll be able to check out this new characteristic by downloading the most recent Android Studio canary. We’ve outlined a couple of use circumstances to attempt, however we’d love to listen to what you assume as we work by way of bringing this characteristic into future steady releases. Test it out:
Picture attachment – a brand new dimension of interplay
We first previewed Gemini’s multimodal capabilities at Google I/O 2024. This know-how permits Gemini in Android Studio to grasp easy wireframes, and remodel them into working Jetpack Compose code.
You may now discover a picture attachment icon within the Gemini chat window. Merely connect JPEG or PNG recordsdata to your prompts and watch Gemini perceive and reply to visible info. We have noticed that photos with robust coloration contrasts yield the most effective outcomes.
We encourage you to experiment with varied prompts and pictures. Listed here are a couple of compelling use circumstances to get you began:
- Fast UI prototyping and iteration: Convert a easy wireframe or high-fidelity mock of your app’s UI into working code.
- Diagram clarification and documentation: Acquire deeper insights into complicated structure or information stream diagrams by having Gemini clarify their elements and relationships.
- UI troubleshooting: Seize screenshots of UI bugs and ask Gemini for options.
Fast UI prototyping and iteration
Gemini’s multimodal help allows you to convert visible designs into purposeful UI code. Merely add your picture and use a transparent immediate. It really works whether or not you are working from your individual sketches or from a designer mockup.
Right here’s an instance immediate: “For this picture supplied, write Android Jetpack Compose code to make a display screen that is as near this picture as doable. Be sure that to incorporate imports, use Material3, and doc the code.” After which you’ll be able to append any particular or extra directions associated to the picture.
For extra complicated UIs, refine your prompts to seize particular performance. For example, when changing a calculator mockup, including “make the interactions and calculations work as you’d anticipate” leads to a completely purposeful calculator:
Be aware: this characteristic gives an preliminary design scaffold. It’s a great “first draft” and your edits and changes shall be wanted. Widespread refinements embrace making certain right drawable imports and importing icons. Think about the generated code a extremely environment friendly place to begin, accelerating your UI growth workflow.
Diagram clarification and documentation
With Gemini’s multimodal capabilities, you can too attempt importing a picture of your diagram and ask for explanations or documentation.
Instance immediate: Add the Now in Android structure diagram and say “Clarify the elements and information stream on this diagram” or “Write documentation about this diagram”.
UI troubleshooting
Leverage Gemini’s visible evaluation to establish and resolve bugs rapidly. Add a screenshot of the problematic UI, and Gemini will analyze the picture and recommend potential options. You can even embrace related code snippets for extra exact help.
Within the instance beneath, we used Compose UI verify and located that the button is stretched too vast in pill screens, so we took a screenshot and requested Gemini for options – it was capable of leverage the window dimension lessons to offer the fitting repair.
Obtain Android Studio right this moment
Obtain the most recent Android Studio canary right this moment to attempt the brand new multimodal options!
As all the time, Google is dedicated to the accountable use of AI. Android Studio will not ship any of your supply code to servers with out your consent. You’ll be able to learn extra on Gemini in Android Studio’s dedication to privateness.
We recognize any suggestions on stuff you like or options you wish to see. If you happen to discover a bug, please report the problem and likewise take a look at recognized points. Keep in mind to additionally observe us on X, Medium, or YouTube for extra Android growth updates!