Even if SerpentAI was installed using python3, the tool invokes wrong python version when running serpent setup if environment pip points to python2 causing dependency problems.

I changed shebang to point to python3 in case python and python3 invoke different python version so this would be in sync with correct python version when installing dependencies using serpent setup (which now uses python3 -m pip instead of pip to avoid chances of version mismatch).

Also updated .gitignore to exclude config files.

Hello! ,

Thanks a lot for the work you've done so far.

Preface:

I've checked the contribute.md file and I fully understand your policy but I will definitely do some works to implements what I need for a personal project on top of your work. So I will probably send random pull request... As those pull requests might appear from nowhere, I'm not expecting you to merge them fast or even at all. BUT if my pull request annoys you, feel free to tell me to stop 👍🏻

Back to the work done:

I needed the capacity to have a little bit more control over the window_controller on OSX. Precisely, I wanted to select a precise window of an app (some emulator creates multiple windows).

So I changed the locate function on OSX for a tuple (str:appName, int:windowNumber) which can be considered an identifier (windowNumber is 1-indexed). OSX keeps an array of created windows for every process, they are added in the order of their creations. As long as one doesn't delete a window, the indices can be considered static. In the different usecases of SerpentAI, this is probably fine.

What do you think?

PS: I'm not a OSX guru at all, so feel free to criticise.

With Linux and Windows support out of the way, I thought I'd start on macOS support. Until now, I can confirm these features work:

• Setting up serpent (serpent setup)
• Grab frames (serpent grab_frames)
• Generating a Game Plugin (serpent generate <game/game_agent>)
• Activate/Deactivate Plugin (serpent activate/deactivate <plugin>)
• List Plugins (serpent plugins)
• Launching a Game (serpent launch <game>)
• Capture frames (serpent capture frame <game>)
• Capture Region (serpent capture region <game> <interval> <region>)
• Capture context (serpent capture context <game> <interval> <name>)
• Train Context (serpent train context <epochs>) with Tensorflow CPU
• Launching Visual Debugger (serpent visual_debugger)
• Play Game with Game Agent (serpent play <game> <game_agent>)

I cannot test for GPU Accelerated Tensorflow since I do not have access to a dedicated graphics card in my laptop. I will continue to work on the macOS support by implementing a full working game agent.

Retina Displays - The difficulty with macOS support is the need to support Retina displays. For now the frame_grabber assumes a user with macOS is always using a Retina display.

if sys.platform == "darwin":
  # Half width / height of frame here

Perhaps this is not the best way to handle it and as such there is an additional option to check for a Retina display.

if sys.platform == "darwin":
  if subprocess.call("system_profiler SPDisplaysDataType | grep 'retina'", shell= True) == 1:
      # Half width / height of frame here

I would love some input as to how to handle this.

The installation process is similar to Linux, but requires some extra dependencies, which have been added via the setup process as well as requirements.darwin.txt. I have also created a macOS Installation Guide for those that want to try it out. I'm willing to maintain the macOS branch.

Gif

• Launching a Game

Screenshots

• Captured Frame
• Train Context
• Running Context Classifier
• Visual Debugger

needs pip install opencv-python fixes https://github.com/SerpentAI/SerpentAI/issues/200 good enough for me; today it doesn't do:

• multi-frame sprites
• regions of interest
• ~~alpha channel/transparency~~

Those first two should be trivial to add back in, ~~not 100% sure about transparency. Should be possible with the mask parameter~~

Validation can take a significant amount of time. This may not be desired for all training.

This introduces the ability to specify if you'd like to validate your model or not.

Fix for #46

I've added the x_offset and y_offset to the click() method while removing all offsets from other methods using the click() method. Now the click() method also includes offsets.

I've also added the offsets to the drag() and scroll() methods.

This is one small step to decoupling from config file and making it CLI friendly. It matches game classes from partial name. Those match functions are lingering in tasks.py and would be better to move somewhere appropriate :) invoke capture_context --game=youmust --context="level_test"

One of the problem was once game.play function is called, it creates agent itself by name and goes into endless loop. Maybe make it take class instance instead of name? I solved this by adding a callback that called right after class is instantiated. Let me know if it doesn't fits taste of the framework :)

Hello.

It really dazzles my eyes when I see invoking "clear" shell command from programming languages. It creates a shell, then it is running the clear command and result. Spawning a shell is unnecessary. We can send a code to output, that clears the visible buffer - which is <ESC>c (in other words 033c).

I also deleted unused imports.

this prevents subprocess.CalledProcessError: Command '['xdotool', 'search', '--onlyvisible', '--name', '^WINDOW TITLE$']' returned non-zero exit status 1. errors on my system when launching games

In old implementation I got crashes when more than half of memory was used by Serpent program. Fork duplicate calling process, so in that case it cannot allocate more memory. (https://stackoverflow.com/questions/1367373/python-subprocess-popen-oserror-errno-12-cannot-allocate-memory)

I wrote CmdRunner class which runs commands with minimal memory usage (Process is started using spawn method instead of fork) and changed LinuxWindowController class, it now use CmdRunner.

…lass; updated game class template; added the concept of game inputs to game APIs; added a way to combine game input axes to game APIs; added first unit tests

I got really excited to put some Reinforcement Learning in practice with my favorite games thanks to SerpentAI. However, I've ran into some problems while trying to make Serpent work (sorry for the emails, Nicholas Brochu).

In order to prevent anyone else to have to deal with those problems, I've opened a fork and added some modified files, mixing the old Serpent codes with the new ones from 2020 version. I also modified some codes in keras that were generating compability errors(since the code was still for keras < 2.0 and tensorflow V1).

I was also having some problems with window controller, since pywin32 wasn't extracting the correct size of my game's window. I added an import pyautogui to fix this problem.

c:\users\micha\anaconda3\envs\serpent\lib\site-packages\offshoot\base.py:38: UserWarning: 'offshoot.yml' not found! Using default configuration. warnings.warn("'offshoot.yml' not found! Using default configuration.") Traceback (most recent call last): File "c:\users\micha\anaconda3\envs\serpent\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\users\micha\anaconda3\envs\serpent\lib\runpy.py", line 85, in run_code exec(code, run_globals) File "C:\Users\micha\anaconda3\envs\serpent\Scripts\serpent.exe_main.py", line 7, in File "c:\users\micha\anaconda3\envs\serpent\lib\site-packages\serpent\serpent.py", line 52, in execute command_function_mappingcommand TypeError: launch() takes 1 positional argument but 3 were given

How do i fix this? Also I can't seem to use the Visual Debugger...