IPhoneMirroringAgent

by instapal-ai

RPA Security

2 forks

4 stars

Free

Connect to Server

iPhoneMirroringAgent Overview

What is iPhoneMirroringAgent

The iPhoneMirroringAgent is an innovative tool developed by instapal-ai designed to automate interactions with a mirrored iPhone screen. This Python-based application leverages the power of Claude AI to seamlessly perform tasks on your mirrored iPhone. By capturing screenshots of your iPhone display, analyzing them through AI, and executing simulated touch operations, it offers a sophisticated solution for managing tasks on your device without direct physical interaction.

How to Use iPhoneMirroringAgent

Getting started with the iPhoneMirroringAgent is straightforward:

Installation: To begin, install the application by setting up the necessary Python environment and dependencies on your system.
Launching the Interface: Run the main script to open the graphical user interface (GUI). This interface is designed for ease of use, providing all the necessary fields to configure your task.
Configuration: Enter your Anthropic API key, select the AI model, and adjust parameters such as the number of tokens and temperature of responses. Define the task you want the agent to perform.
Start the Task: Simply click on the "Start Task" button. The application will automatically bring the iPhone Mirroring app to the forefront, flash it for visibility, and initiate the task.
Automation: The agent captures screenshots of the mirrored screen, sends them to Claude AI for analysis, and executes the resulting instructions. This cycle continues until the task is completed.

Key Features of iPhoneMirroringAgent

AI-Driven Screen Analysis: Uses Claude AI for intelligent analysis of screen captures, enhancing task automation efficiency.
Simulated Interactions: Performs simulated cursor movements and clicks, replicating user actions on the mirrored iPhone.
Real-Time Feedback: The user-friendly GUI offers real-time updates on task progress, ensuring users are informed throughout the process.
Configurable Settings: Users can tailor AI model parameters such as temperature and token limits to fit specific needs.
Compatibility: Works with any iPhone model mirrored through the iPhone Mirroring macOS app, ensuring broad usability.
Conversation Logging: Keeps a record of AI interactions, which is useful for tracking and improving task execution.
Automatic Focus: Automatically flashes the iPhone Mirroring app window to ensure it's the active window when a task starts.

This tool is ideal for anyone looking to automate routine tasks on their iPhone without needing to constantly interact with the device manually. Whether for personal use or professional applications, iPhoneMirroringAgent provides a sophisticated, automated approach to iPhone management, making it a valuable addition to your tech toolkit.

The iPhone Mirroring Agent is a Python-based tool that uses Claude AI to automate interactions with a mirrored iPhone screen. It captures screenshots, analyzes them using Claude AI, and performs simulated touch operations based on AI-generated instructions. iPhone Mirroring Agent Screenshot

Features

Screen capture and analysis using Claude AI
Simulated cursor movements and clicks
Conversation logging
Configurable AI model parameters
User-friendly graphical interface for easy interaction and real-time feedback
Automatic flashing of the iPhone Mirroring app window when starting a task
Compatible with any iPhone model connected through the iPhone Mirroring macOS app

Installation

Clone this repository:

git clone 
cd iPhoneMirroringAgent

Install the required dependencies using the provided requirements.txt file:

pip install -r requirements.txt

Usage

Run the script with the following command:

python main.py

This will launch the graphical user interface. Enter your Anthropic API key, configure the parameters, and provide a task description in the input fields. Click the "Start Task" button to begin the automation process.

Configuration

The application allows you to configure the following parameters through the GUI:

API Key: Your Anthropic API key
Model: The Claude AI model to use (default: "claude-3-5-sonnet-20240620")
Max Tokens: Maximum number of tokens in Claude's response (default: 2048)
Temperature: Temperature for Claude's responses (0.0 to 1.0, default: 0.7)
Max Messages: Maximum number of messages in the conversation (default: 20)
Task Description: The task you want the agent to perform on the mirrored iPhone screen

Project Structure

The project is organized into multiple files for better modularity and maintainability:

main.py: Entry point of the application
gui.py: Contains the MainWindow class and GUI-related code
agent.py: Contains the iPhoneMirroringAgent class for interacting with the iPhone mirroring and Claude API
constants.py: Contains constant values like SYSTEM_PROMPT and TOOLS
screen.py: Contains utility functions for screen capture, window management, and cursor operations

How It Works

When the "Start Task" button is clicked, the application attempts to bring the iPhone Mirroring app window to the front and flash it to draw attention.
The script captures a screenshot of the mirrored iPhone screen.
The screenshot is sent to Claude AI for analysis.
Claude AI provides instructions for the next action (move cursor, click, etc.).
The script executes the instructed action using PyAutoGUI.
This process repeats until the task is completed or the maximum number of messages is reached.
The GUI provides real-time updates on the task progress and displays the current screenshot.

Limitations

The effectiveness of the automation depends on the quality of the mirrored display and the complexity of the task.
The script relies on the Anthropic API, so an active internet connection is required.
The window flashing feature requires the iPhone Mirroring app to be running and visible on the screen.
The AI's ability to interact with the iPhone interface may vary depending on the specific app or task being performed.

Contributing

Contributions to improve the iPhone Mirroring Agent are welcome. Please feel free to submit issues or pull requests.

License

This project is licensed under the MIT License. See the LICENSE file for details.

How to Use

To use the iPhoneMirroringAgent, follow these steps:

Visit https://github.com/instapal-ai/iPhoneMirroringAgenthttps://github.com/instap...
Follow the setup instructions to create an account (if required)
Connect the MCP server to your Claude Desktop application
Start using iPhoneMirroringAgent capabilities within your Claude conversations

Additional Information

Created

July 6, 2024

Company

instapal-ai

Related MCP Servers

Start building your own MCP Server

Interested in creating your own MCP Server? Check out the official documentation and resources.

Learn More