2022-05-05 14:10:28 [scrapy.utils.log] INFO: Scrapy 2.6.1 started (bot: scrapybot)
2022-05-05 14:10:28 [scrapy.utils.log] INFO: Versions: lxml 4.8.0.0, libxml2 2.9.13, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 22.4.0, Python 3.10.4 (main, Mar 24 2022, 13:07:27) [GCC 11.2.0], pyOpenSSL 22.0.0 (OpenSSL 3.0.2 15 Mar 2022), cryptography 37.0.1, Platform Linux-5.17.0-1-amd64-x86_64-with-glibc2.33
2022-05-05 14:10:28 [scrapy.crawler] INFO: Overridden settings:
{}
2022-05-05 14:10:28 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor
2022-05-05 14:10:28 [scrapy.extensions.telnet] INFO: Telnet Password: 91c143582ecd5828
2022-05-05 14:10:28 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats']
2022-05-05 14:10:28 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2022-05-05 14:10:28 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2022-05-05 14:10:28 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2022-05-05 14:10:28 [scrapy.core.engine] INFO: Spider opened
2022-05-05 14:10:28 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2022-05-05 14:10:28 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2022-05-05 14:10:29 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://torrentgalaxy.to/> (failed 1 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'unexpected eof while reading')]>]
2022-05-05 14:10:29 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://torrentgalaxy.to/> (failed 2 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'unexpected eof while reading')]>]
2022-05-05 14:10:29 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying <GET https://torrentgalaxy.to/> (failed 3 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'unexpected eof while reading')]>]
2022-05-05 14:10:29 [scrapy.core.scraper] ERROR: Error downloading <GET https://torrentgalaxy.to/>
Traceback (most recent call last):
  File "/home/aquib/.local/lib/python3.10/site-packages/scrapy/core/downloader/middleware.py", line 49, in process_request
    return (yield download_func(request=request, spider=spider))
twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'unexpected eof while reading')]>]
2022-05-05 14:10:29 [scrapy.core.engine] INFO: Closing spider (finished)
2022-05-05 14:10:29 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/exception_count': 3,
 'downloader/exception_type_count/twisted.web._newclient.ResponseNeverReceived': 3,
 'downloader/request_bytes': 648,
 'downloader/request_count': 3,
 'downloader/request_method_count/GET': 3,
 'elapsed_time_seconds': 0.881984,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2022, 5, 5, 8, 40, 29, 711889),
 'log_count/DEBUG': 3,
 'log_count/ERROR': 2,
 'log_count/INFO': 10,
 'memusage/max': 62857216,
 'memusage/startup': 62857216,
 'retry/count': 2,
 'retry/max_reached': 1,
 'retry/reason_count/twisted.web._newclient.ResponseNeverReceived': 2,
 'scheduler/dequeued': 3,
 'scheduler/dequeued/memory': 3,
 'scheduler/enqueued': 3,
 'scheduler/enqueued/memory': 3,
 'start_time': datetime.datetime(2022, 5, 5, 8, 40, 28, 829905)}
2022-05-05 14:10:29 [scrapy.core.engine] INFO: Spider closed (finished)

getting this problem running python3 main.py and selecting top

Traceback (most recent call last):
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f57355079a0>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.torrentgalaxy.to', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f57355079a0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/elon/stream-cli/main.py", line 130, in <module>
    main()
  File "/home/elon/stream-cli/main.py", line 87, in main
    get_top(hosts)
  File "/home/elon/stream-cli/main.py", line 14, in get_top
    1: apprun(torrent_galaxy()),
  File "/home/elon/stream-cli/stream_cli/runner.py", line 51, in apprun
    movies = start_scrawling(scraping_class)
  File "/home/elon/stream-cli/stream_cli/runner.py", line 35, in start_scrawling
    response = requests.get("https://www.torrentgalaxy.to/").status_code
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/home/elon/stream-cli/venv/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.torrentgalaxy.to', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f57355079a0>: Failed to establish a new connection: [Errno 111] Connection refused'))

using npm 8.15.1; node 18.7.0 and webtorrent 4.1.0 (1.8.26).

love the project concept, thanks in advance

Got this error when running python -m venv venv && source venv/bin/activate.

I'm using windows 10 and running the command using Git Bash console.

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 704, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request self._validate_conn(conn) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1043, in validate_conn conn.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 414, in connect self.sock = ssl_wrap_socket( File "/usr/lib/python3/dist-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket ssl_sock = ssl_wrap_socket_impl( File "/usr/lib/python3/dist-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.10/ssl.py", line 512, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.10/ssl.py", line 1070, in _create self.do_handshake() File "/usr/lib/python3.10/ssl.py", line 1341, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/damn/.local/lib/python3.10/site-packages/requests/adapters.py", line 440, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 786, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 550, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 718, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 704, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request self._validate_conn(conn) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1043, in validate_conn conn.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 414, in connect self.sock = ssl_wrap_socket( File "/usr/lib/python3/dist-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket ssl_sock = ssl_wrap_socket_impl( File "/usr/lib/python3/dist-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.10/ssl.py", line 512, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.10/ssl.py", line 1070, in _create self.do_handshake() File "/usr/lib/python3.10/ssl.py", line 1341, in do_handshake self._sslobj.do_handshake() urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/damn/stream-cli/main.py", line 69, in app() File "/home/damn/.local/lib/python3.10/site-packages/typer/main.py", line 214, in call return get_command(self)(*args, **kwargs) File "/home/damn/.local/lib/python3.10/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/home/damn/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/damn/.local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/damn/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/damn/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/damn/.local/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper return callback(**use_params) # type: ignore File "/home/damn/stream-cli/main.py", line 19, in top apprun(TopMoviesSpider, True) File "/home/damn/stream-cli/src/runner.py", line 43, in apprun movies = start_scrawling(scraping_class) File "/home/damn/stream-cli/src/runner.py", line 30, in start_scrawling response = requests.get("https://www.torrentgalaxy.to/").status_code File "/home/damn/.local/lib/python3.10/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/home/damn/.local/lib/python3.10/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/damn/.local/lib/python3.10/site-packages/requests/sessions.py", line 529, in request resp = self.send(prep, **send_kwargs) File "/home/damn/.local/lib/python3.10/site-packages/requests/sessions.py", line 645, in send r = adapter.send(request, **kwargs) File "/home/damn/.local/lib/python3.10/site-packages/requests/adapters.py", line 501, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

Running on wsl everything seems to work fine, I set up vlc, but a few seconds after choosing a movie it shows me the message "webtorrent is exiting...". I can't seem to stream any movie.... Every requirement is satisfied and vlc is installed. I'm using Xming to get acess to the graphical interface on windows.

There's an issue with the Unable to connect to torrent provider. Please use vpn. Exiting... message. It is displayed even if the crawler didn't find the movie (for example, looking for a fictional movie name).

I suggest adding a ping to the website before displaying the message to check whether it's a connection issue or the movie is not found, then display a message accordingly.

The code to change:

if spider_class.output == []:
        utils.clear_screen()
        sys.exit(
            "\033[91mUnable to connect to torrent provider. Please use vpn. Exiting...\033[0m"
        )

copy pasted every command and in python3 main.py config "vlc" i get

Traceback (most recent call last):
  File "/home/ashroy/Softwares/stream-cli/main.py", line 7, in <module>
    from stream_cli.runner import apprun
ImportError: cannot import name 'apprun' from 'stream_cli.runner' (/home/ashroy/Softwares/stream-cli/stream_cli/runner.py)

This PR contains the following changes:

• Formatting code using black
• Replace direct execution of cls or clear using os module by rich console's clear method
• Replace urllib urlopen with requests.get(url).status_code to get website status code see this
• Improve user input by adding an exit option (0) and keep asking for valid input
• Use rich colors instead of ANSI escape sequences.

This fixes the FileNotFoundError error on Windows when calling the stream function in stream.py file.

I added the shell=True parameter to the subprocess.run function.

basically title, everything works up to the point until I choose an option. The only thing is that when going through the requirments the PyDispactcher gave an error, but I installed it after so when running the requirments everything is satisfied.

after i've watched a movie, where does the file go ? is it deleted or stored in a specific folder ?

Also, THIS script is BEAUTIFUL ! in a long time i haven't seen a piece of python script so awesome, Keep it up !