-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mainstay doc #64
base: main
Are you sure you want to change the base?
Mainstay doc #64
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
# Mainstay integration | ||
|
||
Options for attesting state to a mainstay proof-of-publication service. | ||
|
||
Assumptions: | ||
|
||
Mainstay service is available over http interface (or via SOCKS5 Tor proxy). | ||
Mainstay service is available and funded with a valid `token_id` for verifiation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My understanding - Client buy “publication slot” with a bitcoin payment, gets a credential and then redeem the service at anytime in the future (under max service policy time window) with cleartext credentials and an identifier. The identifier allows binding between the credentials redemption payload and the protocol-specific request. This identifier can be the valid Note this is matching the issuance / redemption flow of the staking credential framework: The There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, this makes sense. |
||
Funding (via LN payment) is performed in advance and out of band for subscrption. (i.e. `token_id` is already performed.) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The funding can happens through the “issuance” protocol flow of staking credentials mentioned above. Pay-per-usage or subscription can be defined as service policy, though for privacy-preserving reasons if subscription is opted-in new credentials / tokens should be refreshed for every service unit deliverance. A user A should not be able to be dissociated from user B based on its service consumption pattern (ideally). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a fundamental issue with mainstay (or any proof-of-publication mechanism or service). The commitments must be provably unique in a given publication space, and so user A must have exclusive access to their own publication space (i.e. 'slot' in mainstay), necessitating user credentials. The credentials can be updated, but the identification of the publication space they are linked to cannot be - the service will always know it's the same user posting commitments to the same slot. But I don't think there is an issue with this privacy-wise. The user can blind the commitments themselves if required, and store the blinding nonces with the proofs for verification. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay I see the provably unique requirement in a proof-of-publication, though on the exclusive access I wonder if a user signature (and therefore posession of a secret key) could be included in the commitment scope. If you have duplication or equivocation of a publication space it can be disregarded at both client / server level. If my understanding of proof-of-publication space is correct. Otherwise yes credential can be re-used indefinitely by the user, like the service provider binds a slot at the first credential redemption, and allow re-use of it. |
||
Mainstay proofs are stored and made available, but that verification against `bitcoind` and staychain occur separately. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One of the nice advantage with current One advantage of dissociating mainstay service from the backend storage is to enable the replication of storage over multiple Storage service requirement will need to be agree on as it can become a source of denial-of-service. I think it’s good than verification against Lastly, I believe it would be very valuable to have standardization of the mainstay proofs, that way it can be consumed by There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, yes. The mainstay model as it currently works is:
The user can then choose either: just trust the mainstay service provider that the tx is confirmed, the proof is valid and done correctly, and just keep the data in case it is needed for future dispute. OR verify the commitment once it's received against bitcoind. In the current service, verification is handled by the pymainstay client. The proof format (i.e. a single slot proof) returned by the API is currently like:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay, the mainstay mode is quite simple and I think it fits well in the civkit service framework. There is just a relay (i.e Good to have proofs that can be queried from service or in by the client (in case of service unavailability). Mainstay proof format is simple, that’s good. |
||
|
||
## Config | ||
|
||
Node is configured with the mainstay server URL, the slot index and the authentication token: | ||
|
||
``` | ||
pub struct MainstayConfig { | ||
url: String, | ||
position: u64, | ||
token: String, | ||
} | ||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think one key element which sounds missing from a mainstay service is a long-term pubkey, ideally using a public key on Bitcoin’s I think the url can stay and it could be announced in the future where is civkit service gossip periodically issued by the Unclear what will be a slot index, like where in a batched mainstay proof this client proof is inserted. Authentication token or credential is assumed to be dynamic thanks to the issuance flow. Other fields that could be added is the list of “mainstay” features supported, though this can become more sophisticated later I think. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK - so the long term pubkey is to receive messages via tcp (as opposed to an onion address). The slot index is unique to a user/client. It is assigned by the mainstay service when a user first pays. The slot index cannot change for a single proof-of-publication. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In fact both, see BOLT4 on how pubkey is used for onion routing: https://github.com/lightning/bolts/blob/master/04-onion-routing.md Understood the slot index unique to a user/client. |
||
|
||
This can be added to `/src/config.rs` | ||
|
||
## Commitment function | ||
|
||
Impliementation of a commitment function that performs a POST request to the `/commitment/send` mainstay service route, with payload: | ||
|
||
``` | ||
payload = { | ||
commitment: commitment, | ||
position: 0, | ||
token: '4c8c006d-4cee-4fef-8e06-bb8112db6314', | ||
}; | ||
``` | ||
|
||
`commitment` is a 32 byte value encoded as a 64 character hex string | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally the payload can be defined as a new I think it’s a bit of protocol hacking though in the future this allow nice thing, like leveraging the nostr tag field to have “mempool” like semantic of relay messages, or extract the |
||
|
||
This can be performed using the `Reqwest` http client library (as in mercury server), e.g. | ||
|
||
``` | ||
use reqwest; | ||
|
||
pub struct Request(reqwest::blocking::RequestBuilder); | ||
|
||
impl Request { | ||
//Construct a request from the give payload and config | ||
pub fn from( | ||
payload: Option<&Payload>, | ||
command: &String, | ||
config: &MainstayConfig, | ||
signature: Option<String>, | ||
) -> Result<Self> { | ||
//Build request | ||
let client = reqwest::blocking::Client::new(); | ||
let url = reqwest::Url::parse(&format!("{}/{}", config.url(), command))?; | ||
|
||
//If there is a payload this is a 'POST' request, otherwise a 'GET' request | ||
let req = match payload { | ||
Some(p) => { | ||
let payload_str = String::from(serde_json::to_string(&p)?); | ||
let payload_enc = encode(payload_str); | ||
let mut data = HashMap::new(); | ||
data.insert("X-MAINSTAY-PAYLOAD", &payload_enc); | ||
let sig_str = match signature { | ||
Some(s) => s, | ||
None => String::from(""), | ||
}; | ||
data.insert("X-MAINSTAY-SIGNATURE", &sig_str); | ||
client | ||
.post(url) | ||
.header(reqwest::header::CONTENT_TYPE, "application/json") | ||
.json(&data) | ||
} | ||
None => client | ||
.get(url) | ||
.header(reqwest::header::CONTENT_TYPE, "application/json"), | ||
}; | ||
|
||
Ok(Self(req)) | ||
} | ||
|
||
pub fn send(self) -> std::result::Result<reqwest::blocking::Response, reqwest::Error> { | ||
self.0.send() | ||
} | ||
} | ||
``` | ||
|
||
## Commitment construction | ||
|
||
The node will construct commitments from specified *events* () in `src/events.rs`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it should be noted the nostr events is just a data communication transport and ideally attestation could scope more generic data payload. Note the events in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK - so we need to define precisely what it is that needs to be committed and proven in a dispute. There's no reason this can't be everything that is stored permanently by the node? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I don’t get exactly your question, like assuming everything is forever stored by the node? I think you have denial-of-service if proofs can be freely stored or freely queried by clients. Even if you have subscription, there is a need to a data limit. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The question is the general one of what data needs to be committed. So all events are saved in the db when To verify, all data that formed the commitment needs to be available. Should we just hash There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Other elements to be saved in the DB:
Yes I think too, though note the trade-off in term of disk denial-of-service. E.g I ask you to store a proof-of-publication with no time limit and not regular payment. It’s less an issue with what data is saved than client-server interactions.
I think we can just have |
||
|
||
The commitment can be simply constructed from the sha256 hash of each event (encoded as a string) similar to: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think effectively it is best to have an API receiving a I think it would be valuable to precise the data format of the attestation, like what is included inside beyond the sha256 e.g block hash / timestamp and service counter-signature. I understand a mainstay proof is the attestation + “included-in-the-chain” proof-of-publication. |
||
|
||
``` | ||
pub fn make_commitment(data: &String) -> (String) { | ||
let mut data_vec = data.as_bytes().iter().cloned().collect::<Vec<u8>>(); | ||
|
||
let commitment = sha256d::Hash::hash(&data_vec); | ||
return (commitment.to_string()); | ||
} | ||
``` | ||
|
||
Will determine which events need to be attested. | ||
|
||
## Commitment compression | ||
|
||
Initially assume every event will be committed to the mainstay service endpoint. | ||
|
||
It may be more efficient to compress several events into a single commitment and then only commit every `commitment_interval`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think as soon as compression is wished, the trade-off of the compression format have to be weighted in, as they leak on the storage / retrieval efficiency / robustness / cost. |
||
|
||
## Proof retreival | ||
|
||
After each commitment, retreive the slot proof from the mainstay server API (call to GET `/commitment/commitment` route with the `commitment` hex string). This will return attestion info (`TxID`) and the slot proof (Merkle proof). | ||
|
||
``` | ||
pub struct Proof { | ||
merkle_root: Commitment, | ||
commitment: Commitment, | ||
ops: Vec<Commitment>, | ||
append: Vec<bool>, | ||
position: u64, | ||
} | ||
``` | ||
|
||
This will need to be stored in a new DB table corresponding to events. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this assumes than the mainstay server look on periodically at There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes - mainstay server can just be queried for txids and proofs as they become available. Checking against There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the question to ask if what types of clients the mainstay integration aims to serve during the few first rollouts. Out of mind, I believe one of the main target is Nostr client (including civkit-sample) and civkit marketd service (notarize all the trade orders received) and a more long-term scale LSPs / Lightning delegated infrastructure (e.g watchtower).
If we’re considering those clients in priority, realistically the interface to prioritize are the following:
Those ones are already wip in civkit-node.
W.r.t to communications between
civkit-notaryd
(i.e either mainstay service proxy or one of its main running process) andcivkitd
there is a tonic interface (civkitservices
) using gRPC over HTTP/2.