stem_http package#

Submodules#

stem_http.client module#

The HTTP client code.

class stem_http.client.TorHttpClient(tor_proxy: ProxyMgr | None = None, proxy_addr: str | None = None, proxy_port: int | None = None, timeout: ClientTimeout = ClientTimeout(total=90, connect=15, sock_read=None, sock_connect=None, ceil_threshold=5), verify_ssl: bool = True)#

Bases: object

An async HTTP client that uses a Tor Proxy.

The Tor proxy should be managed by the tor.ProxyMgr class.

Parameters#

tor_proxytor.ProxyMgr | None, optional: An object that manages a local Tor proxy, by default None. If None, a custom socks proxy addr/port can be passed. If all are None, no proxy is used and this object becomes a simple aiohttp wrapper. Not using the tor_proxy arg means that countries cannot be managed by this object.
proxy_addrstr | None, optional: A custom socks proxy address to use, by default None.
proxy_portint | None, optional: The port that the custom socks proxy uses, by default None.
timeoutaiohttp.ClientTimeout, optional: A custom timeout object for requests, by default aiohttp.ClientTimeout(total=90, connect=15).
verify_sslbool, optional: Whether to verify SSL certificates during requests, by default True.

Properties#

tor_proxytor.ProxyMgr | None: The object that manages the local Tor proxy.
verify_sslbool: Whether requests verify SSL certificates.
managed_torbool: True when tor_proxy is not None. Gates whether managed Tor actions can be taken by this object, like changing countries.
connectoraiohttp.BaseConnector: The connector to use with requests. Can be either BaseConnector or an aiohttp_socks.ProxyConnector.
sessaiohttp.ClientSession: The session object used for all requests made by this object.

async close()#: Close the client session.

async get(url, params: Mapping[str, str] | None = None, cookies: Mapping[str, str | BaseCookie[str] | Morsel[Any]] | Iterable[tuple[str, str | BaseCookie[str] | Morsel[Any]]] | BaseCookie[str] | None = None, headers: Mapping[str, str] | Mapping[istr, str] | CIMultiDict | CIMultiDictProxy | Iterable[tuple[str | istr, str]] | None = None, auth: BasicAuth | None = None, **kwargs)#: Make an HTTP GET request.

async post(url: str, params: Mapping[str, str] | None = None, data: Any = None, json: dict | None = None, cookies: Mapping[str, str | BaseCookie[str] | Morsel[Any]] | Iterable[tuple[str, str | BaseCookie[str] | Morsel[Any]]] | BaseCookie[str] | None = None, headers: Mapping[str, str] | Mapping[istr, str] | CIMultiDict | CIMultiDictProxy | Iterable[tuple[str | istr, str]] | None = None, auth: BasicAuth | None = None, **kwargs)#: Make an HTTP POST request.

async put(url: str, params: Mapping[str, str] | None = None, data: Any = None, json: dict | None = None, cookies: Mapping[str, str | BaseCookie[str] | Morsel[Any]] | Iterable[tuple[str, str | BaseCookie[str] | Morsel[Any]]] | BaseCookie[str] | None = None, headers: Mapping[str, str] | Mapping[istr, str] | CIMultiDict | CIMultiDictProxy | Iterable[tuple[str | istr, str]] | None = None, auth: BasicAuth | None = None, **kwargs)#: Make an HTTP PUT request.

Make an HTTP request - by adding it to the request queue.

Parameters#

methodstr: The HTTP method of the request.
urlstr: The URL to make a request to.
paramsOptional[Mapping[str, str]], optional: Optional URL params to add to the URL, by default None.
dataAny, optional: Arbitrary data to send in the body of the request, by default None.
jsondict | None, optional: An optional JSON payload to include in the body, by default None.
cookiesOptional[aiohttp.typedefs.LooseCookies], optional: Optional cookies to send with the request, by default None.
headersOptional[aiohttp.typedefs.LooseHeaders], optional: Optional headers to send with the request, by default None.
authOptional[aiohttp.BasicAuth], optional: Optionally authenticate the request with this object, by default None.

Returns#

aiohttp.ClientResponse: The response of the request.

Raises#

err

Any Exception raised when making the request. The non-exhaustive list:

RuntimeError
TypeError
ValueError
aiohttp.ClientError
asyncio.TimeoutError

stem_http.tor module#

Manage the Tor proxy.

class stem_http.tor.ProxyMgr(reuse: bool = False, retry: bool = True, country: str | None = None, passw: str | None = None, socks_port: int = 9050, cntrl_port: int = 9051, exit_on_reuse_fail: bool = False, log_file: str | None = '/tmp/tor_log', tor_debug: bool = False)#

Bases: object

Manages a Tor Proxy process attached to the calling process.

May be used as a contextmanager that automatically kills the Tor process when it exits.

You cannot have more than one Tor proxy running on a host at once (at least not without extra config). So this object does not support that. If more than one instance of this object connected to the same prox is running at once, any changes this object makes effect the Tor proxy for all instances.

TODO - Support country changes TODO - Expose a method to open raw sockets through the proxy.

Parameters#

reusebool, optional: Reuse an existing Tor proxy, by default False.
retrybool, optional: Try starting a managed Tor process multiple times until one can start, by default True.
countryOptional[str], optional: The country to use when selecting exit nodes, by default None.
passwOptional[str], optional: An optional password to authenticate to the Tor proxy’s control port, by default None.
socks_portint, optional: The port of the Tor SOCKS proxy to connect to or use in managed processes. Retries will change this port as needed until a valid port is found. By default 9050.
cntrl_portint, optional: The Tor proxy’s control port to connect to, by default 9051.
exit_on_reuse_failbool, optional: Raise an exception if reusing an existing proxy fails. If False, the object tries to start a managed process on the specified ports when reuse fails. By default False.
log_fileOptional[str], option: The file to write tor logs to. By default /tmp/tor_log.
tor_debugbool, options: Whether to configure tor with debug logging, by default False.

Raises#

RuntimeError

When reusing a proxy fails and exit_on_reuse_fail is True.
When there’s already a Tor proxy running on this host or there’s a config error.
When there’s been too many retries while starting a new Tor process.

OSError

When ever there’s an error starting the Tor process. See the log_file for details.

connect_cntrlr()#: Connect to the configured Tor proxy’s Control port.

start(retries: int = 10)#

Start a managed Tor process.

Supports retrying the process if an existing process owns the SOCKS or control ports. This allows multiple instances of this class to run. The number of instances is can be limited by the number of retries.

Parameters#

retriesint, optional: The max number of times to try starting a new process, by default 10. Set this to 0 to disable the retry behavior.

Raises#

RuntimeError: When the max number of retries is reached and no process is started.
OSError: When there’s an error starting the Tor subprocess.

stop()#

Kill the tor process.

Must be done before exiting Python or the tor process may persist. The process is supposed to be killed when Python is killed, but this allows for it to happen gracefully, and with the caller’s control.

Has no effect when reusing an existing Tor proxy (self.reuse == True).

Module contents#

The stem-http Python Package.