Skip to content
This repository has been archived by the owner on Jan 29, 2025. It is now read-only.

Not production ready #14

Closed
2 of 4 tasks
s4w3d0ff opened this issue Apr 17, 2017 · 13 comments
Closed
2 of 4 tasks

Not production ready #14

s4w3d0ff opened this issue Apr 17, 2017 · 13 comments
Assignees
@s4w3d0ff
Labels
bug feature request

Comments

@s4w3d0ff
Copy link
Owner

s4w3d0ff commented Apr 17, 2017 •
edited
Loading

TODO:

  • code is pip installable providing a usable library to create other trade bots
  • add deep learning
  • find a solid way to 'daemonize' the main process for most os
  • create a gui (tkinter?)
@s4w3d0ff s4w3d0ff self-assigned this Apr 17, 2017
@hockeyadc
Copy link

Hey I'm trying to just get pushy.py to work. How do I import the "tools" folder? I am not 100% familiar with Pythons structure on installing from GIT yet.

@s4w3d0ff
Copy link
Owner Author

If you have git and pip installed you should be able to just pip3 install git+https://github.com/s4w3d0ff/marconibot.git (this can be done with just about any git repo that has a setup.py file)

Then to import:

from marconi import tools

# to import pushy:
from marconi import pushy

# to import the push application object that pushy uses:
from marconi.tools.poloniex import push

class myWAMPapp(push.Application):

Pushy is using https://github.com/absortium/poloniex-api under the hood (which uses asyncio and only works on python3)

@hockeyadc
Copy link

hockeyadc commented Jun 14, 2017 via email

@s4w3d0ff
Copy link
Owner Author

I figured out what was going wrong with the imports, but when testing pushy I got this error (seems like the same error as the mongodbTicker.py example):

> pip3 uninstall marconi -y
> git clone https://github.com/s4w3d0ff/marconibot.git
> cd marconibot
marconibot> python3 marconi/pushy.py
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): poloniex.com
DEBUG:urllib3.connectionpool:https://poloniex.com:443 "GET /public?command=returnTicker HTTP/1.1" 200 None
INFO:__main__:Populated markets database with ticker data
INFO:__main__:Subscribed to ticker
Traceback (most recent call last):
  File "marconi/pushy.py", line 55, in <module>
    app.run()
   ..., ...
  File "/home/s4w3d0ff/.local/lib/python3.5/site-packages/aiohttp/client.py", line 636, in __aenter__
    self._resp = yield from self._coro
  File "/home/s4w3d0ff/.local/lib/python3.5/site-packages/aiohttp/client.py", line 399, in _ws_connect
    headers=resp.headers)
aiohttp.client_exceptions.WSServerHandshakeError: 502, message='Invalid response status'

This is an open issue here: absortium/poloniex-api#17

Both the examples in the s4w3d0ff/poloniex repo (uses twisted) and pushy (uses asyncio) was working fine until poloniex started working on their 'backend', something has changed within poloniex that is causing (what seems like) ssl errors (maybe poloniex generated a new ssl cert and we are trying to send an old one?)

I use linux, but windows users have claimed that making adjustments to the registry fixes the issue: s4w3d0ff/python-poloniex#115 (comment)

I personally would not recommend making changes to your registry unless you know for certain that it is required.

I am going to submit a ticket with polo support, hopefully they can fix it on their end or shed some light on the issue.

@hockeyadc
Copy link

hockeyadc commented Jun 14, 2017 via email

@s4w3d0ff
Copy link
Owner Author

s4w3d0ff commented Jun 14, 2017 •
edited
Loading

Are you sure the queuedTicker.py works? I cant get it to work, I'm getting the same handshake errors from the push api.

Within your WAMP application you need to create a MongoClient object.

I'm used update_one() with $set and upsert=True so it fills the database if the db entry isn't there (using the market pair as the '_id' of each entry) and updates the data if the db entry already exists.

The simplest WAMP app would look something like:

from pymongo import MongoClient
from twisted.internet import reactor
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.wamp import ApplicationSession, ApplicationRunner

class TickPitcher(ApplicationSession):
    """ WAMP application """
    
    @inlineCallbacks
    def onJoin(self, details):
        self.db = MongoClient().poloniex['ticker']
        yield self.subscribe(self.onTick, 'ticker')
        print('Subscribed to Ticker')

    def onTick(self, *data):
        self.db.update_one(
            {"_id": data[0]},
            {"$set": {'last': data[1],
                      'lowestAsk': data[2],
                      'highestBid': data[3],
                      'percentChange': data[4],
                      'baseVolume': data[5],
                      'quoteVolume': data[6],
                      'isFrozen': data[7],
                      'high24hr': data[8],
                      'low24hr': data[9]
                      }},
            upsert=True)

    def onDisconnect(self):
        if reactor.running:
            reactor.stop()

Then make another db connection and db.find_one({'_id': 'BTC_LTC'}) to get the data (if the data exists, if not find_one will return None which is what was giving you your initial errors TypeError: 'NoneType' object has no attribute 'getitem').

You could create a function within the queuedTicker that does what onTick does above but with the data from the queue instead of directly from the WAMP (IMO that would be messy, but if it works, it works).

@hockeyadc
Copy link

hockeyadc commented Jun 14, 2017 •
edited
Loading

yep queuedticker works for me. I'm not that great with Python so bear with me here. I'm trying to hack the two python scripts together so I get it working. I'll take a look at what you have above and try to get this working

Edit: Not working :-(

@hockeyadc
Copy link

hockeyadc commented Jun 14, 2017 •
edited
Loading

Ok so actually I think its just printing the same ticker value over and over again, so maybe it's not working 👎 Also it doesn't store anything into the database. Here is what I put together:

'import pymongo
from pymongo import MongoClient
from multiprocessing import Process, Queue
from multiprocessing.dummy import Process as Thread

from twisted.internet import reactor
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.wamp import ApplicationSession, ApplicationRunner

import poloniex


queue = Queue()'

class TickPitcher(ApplicationSession):
    """ WAMP application """
    @inlineCallbacks
    def onJoin(self, details):
        self.db = MongoClient().poloniex['ticker']
        yield self.subscribe(self.onTick, 'ticker')
        print('Subscribed to Ticker')
    def onTick(self, *tick):
        self.db.update_one(
            {"_id": tick[0]},
            {"$set": {'last': tick[1],
                      'lowestAsk': tick[2],
                      'highestBid': tick[3],
                      'percentChange': tick[4],
                      'baseVolume': tick[5],
                      'quoteVolume': tick[6],
                      'isFrozen': tick[7],
                      'high24hr': tick[8],
                      'low24hr': tick[9]
                      }},
            upsert=True)
        queue.put(tick)
    def onDisconnect(self):
        if reactor.running:
            reactor.stop()

class Ticker(object):
    def __init__(self):
        self.ticker = poloniex.Poloniex().returnTicker()
        self._appRunner = ApplicationRunner(
                    u"wss://api.poloniex.com:443", u"realm1"
                    )
        self._appProcess, self._tickThread = None, None
        self._running = False

def __call__(self):
    return self.ticker

def tickCatcher(self):
    print("Catching...")
    while self._running:
        try:
            tick = queue.get(timeout=1)
        except:
            continue
        else:
            self.ticker[tick[0]] = {
                'last':tick[1],
                'lowestAsk':tick[2],
                'highestBid':tick[3],
                'percentChange':tick[4],
                'baseVolume':tick[5],
                'quoteVolume':tick[6],
                'isFrozen':tick[7],
                'high24hr':tick[8],
                'low24hr':tick[9],
                'id':self.ticker[tick[0]]['id']
                }
    print("Done catching...")

def start(self):
    """ Start the ticker """
    print("Starting ticker")
    self._appProcess = Process(
            target=self._appRunner.run,
            args=(TickPitcher,)
            )
    self._appProcess.daemon = True
    self._appProcess.start()
    self._running = True
    print('TICKER: tickPitcher process started')
    self._tickThread = Thread(target=self.tickCatcher)
    self._tickThread.deamon = True
    self._tickThread.start()
    print('TICKER: tickCatcher thread started')

def stop(self):
    """ Stop the ticker """
    print("Stopping ticker")
    self._appProcess.terminate()
    print("Joining Process")
    self._appProcess.join()
    print("Joining thread")
    self._running = False
    self._tickThread.join()
    print("Ticker stopped.")

if __name__ == '__main__':
    import time
    ticker = Ticker()
    ticker.start()
    for i in range(10):
        print(ticker()['BTC_LTC']['last'])
        time.sleep(10)
    ticker.stop()
    print("Done")`

@hockeyadc
Copy link

Hey did you ever hear back from poloniex on this? I'm also willing to change my registry settings I just can't find out what to change. I didn't see the registry file on that other post. Do you still have it?

@s4w3d0ff
Copy link
Owner Author

No, nothing from poloniex yet...

Someone in other post mentioned using regedit.exe s4w3d0ff/python-poloniex#115 (comment), I would not recommend touching your registry especially if you don't know what you are doing, messing with the registry can mess up your system. At least back up your system and/or try it on a VM first.

@hockeyadc
Copy link

Yeah I saw that but they never said what to change lol. I have a virtual box setup running python so I don't have any issues changing registry. Do you know the specifics of what I need to change?

@s4w3d0ff
Copy link
Owner Author

s4w3d0ff commented Aug 2, 2017

lol... I still have not heard from poloniex support about this ssl issue, this repo now uses the websocket-client library to connect to the push api, which seems to be working a lot better than WAMP...

@s4w3d0ff
Copy link
Owner Author

I have successfully created a fully working 'test' bot that uses just the tools from this library, so i am closing this issue. Imports have been cleaned up enough to easily navigate the packages and the structure (somewhat) makes sense.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@s4w3d0ff s4w3d0ff

Labels
bug feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@s4w3d0ff @hockeyadc