From d64f62eea9c642d3c7282392ca057011b8cd6642 Mon Sep 17 00:00:00 2001 From: ZiniuYu Date: Fri, 28 Oct 2022 12:13:57 +0800 Subject: [PATCH 1/5] docs: add instructions for using clip server hosted by jina --- docs/hosting/by-jina.md | 95 +++++++++++++++++++++++++++++++++++++++++ docs/index.md | 3 +- 2 files changed, 97 insertions(+), 1 deletion(-) create mode 100644 docs/hosting/by-jina.md diff --git a/docs/hosting/by-jina.md b/docs/hosting/by-jina.md new file mode 100644 index 000000000..3c85bfdd5 --- /dev/null +++ b/docs/hosting/by-jina.md @@ -0,0 +1,95 @@ +# Hosted by Jina AI + +Just like any other machine learning models, CLIP models have better performance when running on GPU. However, it is not always possible to have a GPU machine at hand, and it could be costly to configure a GPU machine. To make CLIP models more accessible, we provide a hosted service for CLIP models. You can send requests to our hosted service and get the embedding results back. + +An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play or develop your CLIP applications. The server is available for **encoding** and **ranking** tasks. + +Before you start, make sure you have obtained an access token from our [console website](https://console.clip.jina.ai/get_started), or via CLI as described in [this guide](https://docs.jina.ai/jina-ai-cloud/login/#create-a-new-pat) + +```bash +jina auth token create -e +``` + +(by-jina-python)= +## Connect in Python + +We provide two ways to send requests to our hosted service: via gRPCs and via HTTPs. + +| Protocol | Address | +| -------- | ------------------------------- | +| gRPCs | `grpcs://api.clip.jina.ai:2096` | +| HTTPs | `https://api.clip.jina.ai:8443` | + + +To use the service, you need select the protocol by specifying corresponding address in the client. For example, if you want to use gRPCs, you need to specify the address as `grpcs://api.clip.jina.ai:2096`. + +Then, you need to configure the access token in the parameter `credential` of the client: + + +````{tab} via gRPCs + +```{code-block} python +--- +emphasize-lines: 4 +--- +from clip_client import Client + +c = Client( + 'grpcs://api.clip.jina.ai:2096', credential={'Authorization': ''} +) + +r = c.encode( + [ + 'First do it', + 'then do it right', + 'then do it better', + 'https://picsum.photos/200', + ] +) +``` + +```` +````{tab} via HTTPs + +```{code-block} python +--- +emphasize-lines: 4 +--- +from clip_client import Client + +c = Client( + 'https://api.clip.jina.ai:8443', credential={'Authorization': ''} +) + +r = c.encode( + [ + 'First do it', + 'then do it right', + 'then do it better', + 'https://picsum.photos/200', + ] +) +``` + +```` + +(by-jina-curl)= +## Connect using plain HTTP request via `curl` + +You can also send requests to our hosted service using plain HTTP request via `curl` by configuring the access token in the HTTP request header `Authorization` as ``. + + +```{code-block} bash +--- +emphasize-lines: 4 +--- +curl \ +-X POST https://api.clip.jina.ai:8443/post \ +-H 'Content-Type: application/json' \ +-H 'Authorization: ' \ +-d '{"data":[{"text": "First do it"}, + {"text": "then do it right"}, + {"text": "then do it better"}, + {"uri": "https://picsum.photos/200"}], + "execEndpoint":"/"}' +``` diff --git a/docs/index.md b/docs/index.md index fdc21b902..9e5a38816 100644 --- a/docs/index.md +++ b/docs/index.md @@ -200,8 +200,9 @@ user-guides/faq :caption: Hosting :hidden: -hosting/colab +hosting/by-jina hosting/on-jcloud +hosting/colab ``` ```{toctree} From 89b0ffd97c6e71c9b0e5ec96f4925cd3b1eedd4a Mon Sep 17 00:00:00 2001 From: ZiniuYu Date: Fri, 28 Oct 2022 12:22:52 +0800 Subject: [PATCH 2/5] fix: update landing page --- docs/index.md | 97 +++++++++++++++++++++++++++------------------------ 1 file changed, 51 insertions(+), 46 deletions(-) diff --git a/docs/index.md b/docs/index.md index 9e5a38816..04a1e0d1f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -7,34 +7,15 @@ ## Try it! -An always-online server `api.clip.jina.ai` loaded with `ViT-L/14-336px` is there for you to play & test. - -Before you start, make sure you have created access token from our [console website](https://console.clip.jina.ai/get_started), -or CLI as described in [this guide](https://github.com/jina-ai/jina-hubble-sdk#create-a-new-pat). +An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play & test. +Before you start, make sure you have created an access token from our [console website](https://console.clip.jina.ai/get_started), +or via CLI as described in [this guide](https://github.com/jina-ai/jina-hubble-sdk#create-a-new-pat). ```bash jina auth token create -e ``` -Then, you need to set the created token in HTTP request header `Authorization` as ``, -or configure it in the parameter `credential` of the client in python. - - -````{tab} via HTTPS ๐Ÿ” - -```bash -curl \ --X POST https://api.clip.jina.ai:8443/post \ --H 'Content-Type: application/json' \ --H 'Authorization: ' \ --d '{"data":[{"text": "First do it"}, - {"text": "then do it right"}, - {"text": "then do it better"}, - {"uri": "https://picsum.photos/200"}], - "execEndpoint":"/"}' -``` - -```` +Then, you need to configure the access token in the parameter `credential` of the client in python or set it in the HTTP request header `Authorization` as ``. ````{tab} via gRPC โšกโšก @@ -42,11 +23,15 @@ curl \ pip install clip-client ``` -```python +```{code-block} python +--- +emphasize-lines: 5 +--- from clip_client import Client c = Client( - 'grpcs://api.clip.jina.ai:2096', credential={'Authorization': ''} + 'grpcs://api.clip.jina.ai:2096', + credential={'Authorization': ''} ) r = c.encode( @@ -62,11 +47,30 @@ print(r) ```` +````{tab} via HTTPS ๐Ÿ” + +```{code-block} bash +--- +emphasize-lines: 4 +--- +curl \ +-X POST https://api.clip.jina.ai:8443/post \ +-H 'Content-Type: application/json' \ +-H 'Authorization: ' \ +-d '{"data":[{"text": "First do it"}, + {"text": "then do it right"}, + {"text": "then do it better"}, + {"uri": "https://picsum.photos/200"}], + "execEndpoint":"/"}' +``` + +```` + ## Install ![PyPI](https://img.shields.io/pypi/v/clip_client?color=%23ffffff&label=%20) is the latest version. -Make sure you have Python 3.7+. You can install client and server independently. You **don't** have to install both: e.g. installing `clip_server` on a GPU machine and `clip_client` on a local laptop. +Make sure you are using Python 3.7+. You can install the client and server independently. It is **not required** to install both: e.g. you can install `clip_server` on a GPU machine and `clip_client` on a local laptop. ````{tab} Client @@ -112,49 +116,50 @@ pip install "clip_server[tensorrt]" ```` - ## Quick check -After install, you can run the following commands for a quick connectivity check. +After installing, you can run the following commands for a quick connectivity check. ### Start the server -````{tab} Run PyTorch Server +````{tab} Start PyTorch Server ```bash python -m clip_server ``` ```` -````{tab} Run ONNX Server +````{tab} Start ONNX Server ```bash python -m clip_server onnx-flow.yml ``` ```` -````{tab} Run TensorRT Server +````{tab} Start TensorRT Server ```bash python -m clip_server tensorrt-flow.yml ``` ```` -At the first time, it will download the default pretrained model, which may take a minute. Then you will get the following address information: +At the first time starting the server, it will download the default pretrained model, which may take a while depending on your network speed. Then you will get the address information similar to the following: ```text - ๐Ÿ”— Protocol GRPC - ๐Ÿ  Local access 0.0.0.0:51000 - ๐Ÿ”’ Private network 192.168.3.62:51000 - ๐ŸŒ Public address 87.105.159.191:51000 +โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ ๐Ÿ”— Endpoint โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ +โ”‚ ๐Ÿ”— Protocol GRPC โ”‚ +โ”‚ ๐Ÿ  Local 0.0.0.0:51000 โ”‚ +โ”‚ ๐Ÿ”’ Private 192.168.31.62:51000 โ”‚ +| ๐ŸŒ Public 87.105.159.191:51000 | +โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ ``` -It means the server is ready to serve. Note down the three addresses showed above, you will need them later. +This means the server is ready to serve. Note down the three addresses shown above, you will need them later. ### Connect from client ```{tip} Depending on the location of the client and server. You may use different IP addresses: -- Client and server are on the same machine: use local address e.g. `0.0.0.0` -- Client and server are behind the same router: use private network address e.g. `192.168.3.62` -- Server is in public network: use public network address e.g. `87.105.159.191` +- Client and server are on the same machine: use local address, e.g. `0.0.0.0` +- Client and server are connected to the same router: use private network address, e.g. `192.168.3.62` +- Server is in public network: use public network address, e.g. `87.105.159.191` ``` Run the following Python script: @@ -169,11 +174,12 @@ c.profile() will give you: ```text - Roundtrip 16ms 100% -โ”œโ”€โ”€ Client-server network 12ms 75% -โ””โ”€โ”€ Server 4ms 25% - โ”œโ”€โ”€ Gateway-CLIP network 0ms 0% - โ””โ”€โ”€ CLIP model 4ms 100% + Roundtrip 16ms 100% +โ”œโ”€โ”€ Client-server network 8ms 49% +โ””โ”€โ”€ Server 8ms 51% + โ”œโ”€โ”€ Gateway-CLIP network 2ms 25% + โ””โ”€โ”€ CLIP model 6ms 75% +{'Roundtrip': 15.684750003856607, 'Client-server network': 7.684750003856607, 'Server': 8, 'Gateway-CLIP network': 2, 'CLIP model': 6} ``` It means the client and the server are now connected. Well done! @@ -200,7 +206,6 @@ user-guides/faq :caption: Hosting :hidden: -hosting/by-jina hosting/on-jcloud hosting/colab ``` From 45d071b66cf844674405335bde2ee25039d08ccc Mon Sep 17 00:00:00 2001 From: ZiniuYu Date: Fri, 28 Oct 2022 12:24:44 +0800 Subject: [PATCH 3/5] fix: update readme --- README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index aa8978150..88887bcbc 100644 --- a/README.md +++ b/README.md @@ -36,16 +36,15 @@ CLIP-as-service is a low-latency high-scalability service for embedding images a ## Try it! -An always-online server `api.clip.jina.ai` loaded with `ViT-L/14-336px` is there for you to play & test. +An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play & test. Before you start, make sure you have created access token from our [console website](https://console.clip.jina.ai/get_started), -or CLI as described in [this guide](https://github.com/jina-ai/jina-hubble-sdk#create-a-new-pat). +or via CLI as described in [this guide](https://docs.jina.ai/jina-ai-cloud/login/#create-a-new-pat). ```bash jina auth token create -e ``` -Then, you need to set the created token in HTTP request header `Authorization` as ``, -or configure it in the parameter `credential` of the client in python. +Then, you need to configure the access token in the parameter `credential` of the client in python or set it in the HTTP request header `Authorization` as ``. โš ๏ธ Our demo server `demo-cas.jina.ai` is sunset and no longer available after **15th of Sept 2022**. From 4dba9790b9b10b1e7459c828c2ecc298e29694aa Mon Sep 17 00:00:00 2001 From: ZiniuYu Date: Fri, 28 Oct 2022 12:38:57 +0800 Subject: [PATCH 4/5] fix: toctree --- docs/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.md b/docs/index.md index 04a1e0d1f..052439218 100644 --- a/docs/index.md +++ b/docs/index.md @@ -206,6 +206,7 @@ user-guides/faq :caption: Hosting :hidden: +hosting/by-jina hosting/on-jcloud hosting/colab ``` From db314715eda28a6357b8add571fd9a8bceaff8c1 Mon Sep 17 00:00:00 2001 From: ZiniuYu Date: Fri, 28 Oct 2022 14:49:58 +0800 Subject: [PATCH 5/5] fix: address comment --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 88887bcbc..2977bdf16 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ CLIP-as-service is a low-latency high-scalability service for embedding images a ## Try it! An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play & test. -Before you start, make sure you have created access token from our [console website](https://console.clip.jina.ai/get_started), +Before you start, make sure you have obtained an access token from our [console website](https://console.clip.jina.ai/get_started), or via CLI as described in [this guide](https://docs.jina.ai/jina-ai-cloud/login/#create-a-new-pat). ```bash