🔖 Release 3.7.0 (#132)

**Added** - TransferProgress tracking in Response when downloading using `stream=True` based on the Content-Length. (#127) There's no easy way to track the "real" amount of bytes consumed using "iter_content" when the remote is sending a compressed body. This change makes it possible to track the amount of bytes consumed. The `Response` object now contain a property named `download_progress` that is either `None` or a `TransferProgress` object. - HTTP/2 with prior knowledge over TLS or via an unencrypted connection. `disable_http1` toggle is now available through your `Session` constructor. In consequence, you may leverage all HTTP/2 capabilities like multiplexing using a plain (e.g. non-TLS) socket. You may enable/disable any protocols per Session object (but not all of them at once!). In non-TLS connections, you have to keep one of HTTP/1.1 or HTTP/2 enabled. Otherwise, one of HTTP/1.1, HTTP/2 or HTTP/3. A `RuntimeError` may be thrown if no protocol can be used in a given context. **Changed** - Relax main API constraint in get, head, options and delete methods / functions by accepting kwargs. - urllib3-future lower bound version is raised to 2.8.900
jawah · Jun 24, 2024 · d83ab6b · d83ab6b
1 parent e43242a
commit d83ab6b
Show file tree

Hide file tree

Showing 10 changed files with 187 additions and 7 deletions.
diff --git a/HISTORY.md b/HISTORY.md
@@ -1,6 +1,26 @@
 Release History
 ===============
 
+3.7.0 (2024-06-24)
+------------------
+
+**Added**
+- TransferProgress tracking in Response when downloading using `stream=True` based on the Content-Length. (#127)
+  There's no easy way to track the "real" amount of bytes consumed using "iter_content" when the remote is
+  sending a compressed body. This change makes it possible to track the amount of bytes consumed.
+  The `Response` object now contain a property named `download_progress` that is either `None` or a `TransferProgress` object.
+- HTTP/2 with prior knowledge over TLS or via an unencrypted connection.
+  `disable_http1` toggle is now available through your `Session` constructor.
+  In consequence, you may leverage all HTTP/2 capabilities like multiplexing using a plain (e.g. non-TLS) socket.
+  You may enable/disable any protocols per Session object (but not all of them at once!).
+  In non-TLS connections, you have to keep one of HTTP/1.1 or HTTP/2 enabled.
+  Otherwise, one of HTTP/1.1, HTTP/2 or HTTP/3. A `RuntimeError` may be thrown if no protocol can be used in a
+  given context.
+
+**Changed**
+- Relax main API constraint in get, head, options and delete methods / functions by accepting kwargs.
+- urllib3-future lower bound version is raised to 2.8.900
+
 3.6.7 (2024-06-19)
 ------------------
 

diff --git a/README.md b/README.md
@@ -39,6 +39,7 @@ Niquests, is the “**Safest**, **Fastest[^10]**, **Easiest**, and **Most advanc
 | `Direct HTTP/3 Negotiation`         |  ✅[^9]   |  N/A[^8]  |    N/A[^8]    | N/A[^8]       |
 | `Happy Eyeballs`                    |    ✅     |     ❌     |       ❌       | ✅             |
 | `Package / SLSA Signed`             |    ✅     |     ❌     |       ❌       | ✅             |
+| `HTTP/2 with prior knowledge (h2c)` |    ✅     |     ❌     |       ✅       | ❌             |
 </details>
 
 <details>
@@ -144,6 +145,7 @@ Niquests is ready for the demands of building scalable, robust and reliable HTTP
 - Basic & Digest Authentication
 - Familiar `dict`–like Cookies
 - Network settings fine-tuning
+- HTTP/2 with prior knowledge
 - Object-oriented headers
 - Multi-part File Uploads
 - Chunked HTTP Requests

diff --git a/docs/user/advanced.rst b/docs/user/advanced.rst
@@ -1299,21 +1299,37 @@ by passing a custom ``QuicSharedCache`` instance like so::
 
 When the cache is full, the oldest entry is removed.
 
-Disable HTTP/2, and/or HTTP/3
+Disable HTTP/1.1, HTTP/2, and/or HTTP/3
 -----------------------------
 
 You can at your own discretion disable a protocol by passing ``disable_http2=True`` or
 ``disable_http3=True`` within your ``Session`` constructor.
 
-.. warning:: It is actually forbidden to disable HTTP/1.1 as the underlying library (urllib3.future) does not permit it for now.
-
 Having a session without HTTP/2 enabled should be done that way::
 
     import niquests
 
     session = niquests.Session(disable_http2=True)
 
 
+HTTP/2 with prior knowledge
+---------------------------
+
+Interacting with a server over plain text using the HTTP/2 protocol must be done by
+disabling HTTP/1.1 entirely, so that Niquests knows that you know in advance what the remote is capable of.
+
+Following this example::
+
+    import niquests
+
+    session = niquests.Session(disable_http1=True)
+    r = session.get("http://my-special-svc.local")
+    r.version  # 20 (aka. HTTP/2)
+
+.. note:: You may do the same for servers that do not support the ALPN extension for https URLs.
+
+.. warning:: Disabling HTTP/1.1 and HTTP/2 will raise an error (RuntimeError) for non https URLs! As HTTP/3 is designed for the QUIC layer, which itself is based on TLS 1.3.
+
 Thread Safety
 -------------
 
@@ -1417,3 +1433,40 @@ Here is a simple example::
     session.get("https://pie.dev/get", verify="sha256_8fff956b66667ffe5801c8432b12c367254727782d91bc695b7a53d0b512d721")
 
 .. warning:: Supported fingerprinting algorithms are sha256, and sha1. The prefix is mandatory.
+
+TLS Fingerprint (like JA3)
+--------------------------
+
+Some of you seems to be interested in that topic, at least according to the statistics presented to me.
+Niquests is dedicated to providing a software that present a unique and close enough signature (against modern browser)
+that you should be protected against TLS censorship / blocking technics.
+
+We are actively working toward a way to permanently improving this.
+To help us fighting toward the greater good, feel free to sponsor us and/or speaking out loud about your
+experiences, whether about a specific country practices or global ISP/Cloud provider ones.
+
+.. note:: If you are getting blocked, come and get in touch with us through our Github issues.
+
+Tracking the real download speed
+--------------------------------
+
+In a rare case, you may be left with no clue on what is the real "download speed" due to the
+remote server applying a "transfer-encoding" or also know as compressing (zstd, br or gzip).
+
+Niquests automatically decompress response bodies, so doing a call to ``iter_content`` is not going to yield
+the size actually extracted from the socket but rather from the decompressor algorithm.
+
+To remediate this issue we've implemented a new property into your ``Response`` object. Named ``download_progress``
+that is a ``TransferProgress`` instance.
+
+.. warning:: This feature is enabled when ``stream=True``.
+
+Here is a basic example of how you would proceed::
+
+    import niquests
+
+    with niquests.Session() as s:
+        with s.get("https://ash-speed.hetzner.com/100MB.bin", stream=True) as r:
+            for chunk in r.iter_content():
+                # do anything you want with chunk
+                print(r.download_progress.total)  # this actually contain the amt of bytes (raw) downloaded from the socket.
diff --git a/pyproject.toml b/pyproject.toml
@@ -41,7 +41,7 @@ dynamic = ["version"]
 dependencies = [
     "charset_normalizer>=2,<4",
     "idna>=2.5,<4",
-    "urllib3.future>=2.7.905,<3",
+    "urllib3.future>=2.8.900,<3",
     "wassima>=1.0.1,<2",
     "kiss_headers>=2,<4",
 ]

diff --git a/src/niquests/__version__.py b/src/niquests/__version__.py
@@ -9,9 +9,9 @@
 __url__: str = "https://niquests.readthedocs.io"
 
 __version__: str
-__version__ = "3.6.7"
+__version__ = "3.7.0"
 
-__build__: int = 0x030607
+__build__: int = 0x030700
 __author__: str = "Kenneth Reitz"
 __author_email__: str = "[email protected]"
 __license__: str = "Apache-2.0"

diff --git a/src/niquests/_async.py b/src/niquests/_async.py
@@ -117,6 +117,7 @@ def __init__(
         quic_cache_layer: CacheLayerAltSvcType | None = None,
         retries: RetryType = DEFAULT_RETRIES,
         multiplexed: bool = False,
+        disable_http1: bool = False,
         disable_http2: bool = False,
         disable_http3: bool = False,
         disable_ipv6: bool = False,
@@ -177,6 +178,7 @@ def __init__(
         #: Bind to address/network adapter
         self.source_address = source_address
 
+        self._disable_http1 = disable_http1
         self._disable_http2 = disable_http2
         self._disable_http3 = disable_http3
 
@@ -797,6 +799,7 @@ async def get(
         verify: TLSVerifyType = ...,
         stream: Literal[False] = ...,
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> Response: ...
 
     @typing.overload  # type: ignore[override]
@@ -815,6 +818,7 @@ async def get(
         verify: TLSVerifyType = ...,
         stream: Literal[True],
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> AsyncResponse: ...
 
     async def get(  # type: ignore[override]
@@ -832,6 +836,7 @@ async def get(  # type: ignore[override]
         verify: TLSVerifyType = True,
         stream: bool = False,
         cert: TLSClientCertType | None = None,
+        **kwargs: typing.Any,
     ) -> Response | AsyncResponse:
         return await self.request(  # type: ignore[call-overload,misc]
             "GET",
@@ -847,6 +852,7 @@ async def get(  # type: ignore[override]
             verify=verify,
             stream=stream,
             cert=cert,
+            **kwargs,
         )
 
     @typing.overload  # type: ignore[override]
@@ -865,6 +871,7 @@ async def options(
         verify: TLSVerifyType = ...,
         stream: Literal[False] = ...,
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> Response: ...
 
     @typing.overload  # type: ignore[override]
@@ -883,6 +890,7 @@ async def options(
         verify: TLSVerifyType = ...,
         stream: Literal[True],
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> AsyncResponse: ...
 
     async def options(  # type: ignore[override]
@@ -900,6 +908,7 @@ async def options(  # type: ignore[override]
         verify: TLSVerifyType = True,
         stream: bool = False,
         cert: TLSClientCertType | None = None,
+        **kwargs: typing.Any,
     ) -> Response | AsyncResponse:
         return await self.request(  # type: ignore[call-overload,misc]
             "OPTIONS",
@@ -915,6 +924,7 @@ async def options(  # type: ignore[override]
             verify=verify,
             stream=stream,
             cert=cert,
+            **kwargs,
         )
 
     @typing.overload  # type: ignore[override]
@@ -933,6 +943,7 @@ async def head(
         verify: TLSVerifyType = ...,
         stream: Literal[False] = ...,
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> Response: ...
 
     @typing.overload  # type: ignore[override]
@@ -951,6 +962,7 @@ async def head(
         verify: TLSVerifyType = ...,
         stream: Literal[True],
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> AsyncResponse: ...
 
     async def head(  # type: ignore[override]
@@ -968,6 +980,7 @@ async def head(  # type: ignore[override]
         verify: TLSVerifyType = True,
         stream: bool = False,
         cert: TLSClientCertType | None = None,
+        **kwargs: typing.Any,
     ) -> Response | AsyncResponse:
         return await self.request(  # type: ignore[call-overload,misc]
             "HEAD",
@@ -983,6 +996,7 @@ async def head(  # type: ignore[override]
             verify=verify,
             stream=stream,
             cert=cert,
+            **kwargs,
         )
 
     @typing.overload  # type: ignore[override]
@@ -1241,6 +1255,7 @@ async def delete(
         verify: TLSVerifyType = ...,
         stream: Literal[False] = ...,
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> Response: ...
 
     @typing.overload  # type: ignore[override]
@@ -1259,6 +1274,7 @@ async def delete(
         verify: TLSVerifyType = ...,
         stream: Literal[True],
         cert: TLSClientCertType | None = ...,
+        **kwargs: typing.Any,
     ) -> AsyncResponse: ...
 
     async def delete(  # type: ignore[override]
@@ -1276,6 +1292,7 @@ async def delete(  # type: ignore[override]
         verify: TLSVerifyType = True,
         stream: bool = False,
         cert: TLSClientCertType | None = None,
+        **kwargs: typing.Any,
     ) -> Response | AsyncResponse:
         return await self.request(  # type: ignore[call-overload,misc]
             "DELETE",
@@ -1291,6 +1308,7 @@ async def delete(  # type: ignore[override]
             verify=verify,
             stream=stream,
             cert=cert,
+            **kwargs,
         )
 
     async def gather(self, *responses: Response, max_fetch: int | None = None) -> None:  # type: ignore[override]

diff --git a/src/niquests/adapters.py b/src/niquests/adapters.py
@@ -313,6 +313,7 @@ class HTTPAdapter(BaseAdapter):
         "_pool_maxsize",
         "_pool_block",
         "_quic_cache_layer",
+        "_disable_http1",
         "_disable_http2",
         "_disable_http3",
         "_source_address",
@@ -327,7 +328,9 @@ def __init__(
         pool_maxsize: int = DEFAULT_POOLSIZE,
         max_retries: RetryType = DEFAULT_RETRIES,
         pool_block: bool = DEFAULT_POOLBLOCK,
+        *,  # todo: revert if any complaint about it... :s
         quic_cache_layer: CacheLayerAltSvcType | None = None,
+        disable_http1: bool = False,
         disable_http2: bool = False,
         disable_http3: bool = False,
         max_in_flight_multiplexed: int | None = None,
@@ -359,6 +362,7 @@ def __init__(
         self._pool_maxsize = pool_maxsize
         self._pool_block = pool_block
         self._quic_cache_layer = quic_cache_layer
+        self._disable_http1 = disable_http1
         self._disable_http2 = disable_http2
         self._disable_http3 = disable_http3
         self._resolver = resolver
@@ -379,6 +383,8 @@ def __init__(
 
         disabled_svn = set()
 
+        if disable_http1:
+            disabled_svn.add(HttpVersion.h11)
         if disable_http2:
             disabled_svn.add(HttpVersion.h2)
         if disable_http3:
@@ -412,6 +418,8 @@ def __setstate__(self, state):
 
         disabled_svn = set()
 
+        if self._disable_http1:
+            disabled_svn.add(HttpVersion.h11)
         if self._disable_http2:
             disabled_svn.add(HttpVersion.h2)
         if self._disable_http3:
@@ -1284,6 +1292,7 @@ class AsyncHTTPAdapter(AsyncBaseAdapter):
         "_pool_maxsize",
         "_pool_block",
         "_quic_cache_layer",
+        "_disable_http1",
         "_disable_http2",
         "_disable_http3",
         "_source_address",
@@ -1298,7 +1307,9 @@ def __init__(
         pool_maxsize: int = DEFAULT_POOLSIZE,
         max_retries: RetryType = DEFAULT_RETRIES,
         pool_block: bool = DEFAULT_POOLBLOCK,
+        *,
         quic_cache_layer: CacheLayerAltSvcType | None = None,
+        disable_http1: bool = False,
         disable_http2: bool = False,
         disable_http3: bool = False,
         max_in_flight_multiplexed: int | None = None,
@@ -1331,6 +1342,7 @@ def __init__(
         self._pool_maxsize = pool_maxsize
         self._pool_block = pool_block
         self._quic_cache_layer = quic_cache_layer
+        self._disable_http1 = disable_http1
         self._disable_http2 = disable_http2
         self._disable_http3 = disable_http3
         self._resolver = resolver
@@ -1350,6 +1362,8 @@ def __init__(
 
         disabled_svn = set()
 
+        if disable_http1:
+            disabled_svn.add(HttpVersion.h11)
         if disable_http2:
             disabled_svn.add(HttpVersion.h2)
         if disable_http3:
@@ -1383,6 +1397,8 @@ def __setstate__(self, state):
 
         disabled_svn = set()
 
+        if self._disable_http1:
+            disabled_svn.add(HttpVersion.h11)
         if self._disable_http2:
             disabled_svn.add(HttpVersion.h2)
         if self._disable_http3: