Problem
I’ve been working on a program that should find the color red on the screen and then send out an alert. The issue is that the red color moves around a lot and the color finder moves too slow on the screen to find the color accuritely and in time before the color dissapears to another location on the screen.
Is there a way to speed up this color finding process so that it is practically instant?
Here is the code:
ColorFinder.java
import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;
import java.awt.Color;
import java.util.Scanner;
import java.awt.geom.*;
import java.awt.event.*;
import java.awt.*;
public class ColorFinder
{
public static void main(String[] args) throws AWTException
{
//Create memories
Scanner scanner = new Scanner(System.in);
Robot robot = new Robot();
String ans;
int red;
int green;
int blue;
int x=535;
int y=168;
//Prints instructions
System.out.println("Type 'p' to start.");
System.out.println("");
//Waits for start input
ans = scanner.nextLine();
if(ans.equals("p"))
{
//Starts loop
for(int i=0; i<100; i++)
{
i=0;
System.out.println("");
//Color searcher
x=x+1;
if(x>1378 && y>468) //reset on highest values x & y
{
x=535; y=168;
}
if(x>1378) //step down by one pixel with y
{
y=y+1;
x=535;
}
//Get RGB value on screen coords
System.out.println("x:"+x+" y:"+y);//this part is not needed, it's only for visualising if it's working or at what speed it moves
Color color = robot.getPixelColor(x, y);
red=color.getRed();
green=color.getGreen();
blue=color.getBlue();
/*
System.out.println("Red = " + red);
System.out.println("Green = " + green);
System.out.println("Blue = " + blue);
*/
//Remove the note marks below to see how fast it moves
/*
robot.mouseMove(x, y);
if(x==1000)
{break;}
*/
//if it finds the specific red color it stops the loop and prints out "Found red color!"
if(red==255 && green==20 && blue==25)
{
System.out.println("Found red color!");
break;
}
}
}
}
}
I will gladly answer any questions.
Any kind of solution that does the same thing but way faster is good, many thanks in advance!
Solution
This question brings back memories from 2010 when I asked how to draw constantly changing grapics
https://stackoverflow.com/questions/3742731/java-how-to-draw-constantly-changing-graphics
The idea is that you prepare a location rectangle of the area you want to capture and ask the robot to fill it:
Rectangle rect = new Rectangle(p.x - 4, p.y - 4, 8, 8);
final BufferedImage capture = robot.createScreenCapture(rect);
then you loop over the area checking for the match
for (int x = 0; x < 8; x++) {
for (int y = 0; y < 8; y++) {
final Color pixelColor = new Color(capture.getRGB(x, y));
if (!pixelColor.equals(view.getColorAt(x, y))) {
// do something
}
}
}
Typically my computer vision AI’s does not use 64-pixel area, but 64*64=4096, but you can use 4k screenshot has 3840×2160=8294400 pixels and a minimum of 8 million comparisons will actually take some time unless you parallelize the work.
I write c#, so this is written from the head as I remember Java +10 years ago.
Color red = new Color(255, 20, 25);
Rectangle rect = new Rectangle(535, 168, 843, 300);
final BufferedImage capture = robot.createScreenCapture(rect);
Pair<String, Integer> location = FindColor(capture, red);
if(location != null){
System.out.printf("Found red color at (x:{%s}, y: {%s})%n",
location.getValue(0),
location.getValue(1));
where FindColor is something in the lines of:
public Pair<String, Integer> FindColor(BufferedImage capture, Color color){
if (capture == null || color == null)
return null;
for (int x = 0; x < capture.getWidth(); x++)
for (int y = 0; y < capture.getHeight(); y++)
if (new Color(capture.getRGB(x, y)).equals(color))
return Pair.with(x, y);
return null;
}
instead of tuple-like Pair<>, you can use point2d or a similar struct just likely you do not need the null checks.
The bottleneck in your code is mostly because you use a method (java.awt.Robot#getPixelColor
) in a loop to fetch the color pixel by pixel; this method uses lower system API to fetch the color and that takes more time.
You could make the code faster by using the java.awt.Robot#createScreenCapture
to create a screenshot of a certain region and then, iterate on the colors by using the java.awt.image.BufferedImage#getRGB
. The only downside is that the method returns an integer containing the RGB instead of a color object, you will need to convert it to a color.
To convert the RGB to single color, you can use the java.awt.image.ColorModel
to read the RGB value and choose the different color individually.