microsoft / ccf-app-samples Goto Github PK
View Code? Open in Web Editor NEWSample applications for the Confidential Consortium Framework (CCF)
License: MIT License
Sample applications for the Confidential Consortium Framework (CCF)
License: MIT License
Based on data reporting ADR, implement and unit test data data reporting APIs:
Domain Model according to this ADR
Reference these files:
make unit-test
)After repository structure is defined, create a hello world skeleton app for our data-reconciliation scenario.
Includes:
Acceptance Criteria
Currently we have different scripts between docker and the sandbox.sh - We can probably do better here to open the network and then have the same files that test the application. How we start the network should be the only difference.
Our Skelton App will be a hello world app w/ API endpoint!
Set up CI to build app.
There is a need to define the testing strategy inside the sample
Update the documents so that they "flow" better as a new developer to the repository
When a build of main fails (timeout) it does not delete the resource group and we then encounter quota issues
Install the following features in the DevContainer for ease of development
Describe the bug
I get an error of terminating with uncaught exception of type std::logic_error: SGX enclaves are not supported in current build - cannot launch /usr/lib/ccf/libjs_generic.virtual.so
when trying to run an example with the docker virtual image
Steps to reproduce
I had to update the docker file to generate the member cert in the docker image since I can't install ccf locally but I don't think that would cause such an error.
diff --git a/banking-app/docker/ccf_app_js.virtual b/banking-app/docker/ccf_app_js.virtual
index 2b9d6e6..a115605 100644
--- a/banking-app/docker/ccf_app_js.virtual
+++ b/banking-app/docker/ccf_app_js.virtual
@@ -10,11 +10,12 @@ COPY --from=builder /opt/ccf_virtual/lib/libjs_generic.virtual.so /usr/lib/ccf
# copy configuration into image
COPY ./constitution/*.js /app/
COPY ./config/cchost_config_virtual_js.json /app/
-COPY ./workspace/docker_certificates/member0_cert.pem /app/
-COPY ./workspace/docker_certificates/member0_enc_pubk.pem /app/
WORKDIR /app/
+COPY --from=builder /opt/ccf_virtual/bin/keygenerator.sh /app/
+RUN /app/keygenerator.sh --name member0 --gen-enc-key
+
EXPOSE 8080/tcp
CMD ["/usr/bin/cchost", "--config", "/app/cchost_config_virtual_js.json"]
diff --git a/build_image.sh b/build_image.sh
index 9d9b9c9..51c7ca4 100755
--- a/build_image.sh
+++ b/build_image.sh
@@ -24,9 +24,9 @@ mkdir -p $app_path/workspace/docker_certificates
echo "-- generating member0 certificates"
# This is directly related to the member described in host config file
-cd $app_path/workspace/docker_certificates
-$ccf_prefix/keygenerator.sh --name member0 --gen-enc-key
+#cd $app_path/workspace/docker_certificates
+#$ccf_prefix/keygenerator.sh --name member0 --gen-enc-key
2022-12-08T19:48:25.962404Z 100 [info ] ../src/host/main.cpp:125 | CCF version: ccf-3.0.1
2022-12-08T19:48:25.962602Z 100 [info ] ../src/host/main.cpp:133 | Configuration file /app/cchost_config_virtual_js.json:
{
"enclave": {
"file": "/usr/lib/ccf/libjs_generic.virtual.so",
"type": "Virtual"
},
"network": {
"node_to_node_interface": {
"bind_address": "172.17.0.3:8081"
},
"rpc_interfaces": {
"main_interface": {
"bind_address": "172.17.0.3:8080"
}
}
},
"command": {
"type": "Start",
"service_certificate_file": "/app/service_cert.pem",
"start": {
"constitution_files": [
"/app/validate.js",
"/app/apply.js",
"/app/resolve.js",
"/app/actions.js"
],
"members": [
{
"certificate_file": "/app/member0_cert.pem",
"encryption_public_key_file": "/app/member0_enc_pubk.pem"
}
]
}
}
}
2022-12-08T19:48:25.962649Z 100 [info ] ../src/host/main.cpp:164 | Recovery threshold unset. Defaulting to number of initial consortium members with a public encryption key (1).
terminating with uncaught exception of type std::logic_error: SGX enclaves are not supported in current build - cannot launch /usr/lib/ccf/libjs_generic.virtual.so```
The PG have several other groups that are collaborating with CCF in the same capacity as Crew Neutrino. They are currently using AzDO, duplicating issues, over-complex collaboration model. Amaury really likes how we set up our collaboration model: separate, self-managed project, triage --> issues, and how our issues flow into their backlog.
They asked if we could document how we set up this collaboration model.
Acceptance Criteria:
Research LEI data structure + data flow. After investigating if possible, create a sample data set (1 excel spreadsheet).
Acceptance Criteria
Dependent on conversation with Brent
Investigate how we are going to deploy an application to the managed CCF in Azure
Investigate whether there is a bicep possibility with this (escape hatches)
We have existing scripts for the banking-app
Acceptance Criteria:
Bring the banking-app implementation from https://github.com/takuro-sato/ccf-app-template/tree/example_apps .
To confirm the make demo
finishes with exit code 0. It will prevent bugs like #90
Initial Idea:
Fix up --no-default-constitution within prepare_demo.sh
As part of the Banking Application Sample we copy over the constitution/resolve.js so that the sandbox.sh works - is this correct - what is the correct model here?
Next Steps (Nov 9th):
Update after talking to Takuro
When the Banking Sample was originally created it used the constitution of the Sandbox and overrode the resolve.js. We are going to amend the Banking Sample to run the network in docker and not use sandbox.sh. We will also demonstrate the Governance stage and therefore should define the constitution. This would mean we would not use the sandbox files, but rather include all 4 js files in the Banking Sample.
Update 16th November
Q: How do we update a constitution?
A: There is an api that allows you to update the constitution. Just be aware that any outstanding proposals will be closed when a new constitution is approved.
Build the Banking Application in a continuous integration so we can test the validity of any code changes
I have a change that I have made to the sandbox.sh in the main CCF repository. In order to get this we will have to upgrade the samples dev container. This could cause some issues as we move from 3 to 3.1.
The current demo.sh is a mixture of calling the governance APIs and then the actually Banking Application. Split this into 2 separate scripts so that it is easier to understand.
Also add comments in the bash scripts so dev understands what each line of code does.
Current Docker Image builder generates member0 certificate and keys to be used by application. For security reasons, image must not hold that information.
Change build structure to generate member0 certificate in a previous stage, and create docker image only with the relevant information needed to open the network
Next Steps (Nov 9th):
Work with Brent to determine any schema requirements (input and reporting) as well as any demo expectations.
Acceptance Criteria:
https://marketplace.visualstudio.com/items?itemName=vsls-contrib.codetour
ccf-app-template has one for the C++ app
Describe the bug
make test-docker-virtual
doesn't finish if it's run in codespace.
Steps to reproduce
Based on data ingest ADR, implement and unit test data ingest APIs:
Dependent on #83
Reference these files: -
make unit-test
)After cloning the repository, there should be a one line command to test the sample application(s)
Context from PG/Amaury:
We should have a simple way of building a VM in Azure and deploy the sample application to it so we can show customers how to deploy it to Azure
Feedback/Context from PG:
There is some value in this story.
This may be useful for a developer to understand how to get a single node, CCF network app to run on Infrastructure; however, this is not how most will set up their networks in production environments
This may also be useful to run performance testing on deployed/running sample app.
Useful to get someone started
In reality, SGX is not the easiest VM for any dev to create (limited to certain regions, very expensive). Suggested that we leverage any D.C Intel VM for this story.
What may be "More Useful"and provide more value is taking a sample app CCF app here and deploying it to Azure leveraging mCCF ARM templates + documenting that workflow for a developer (more realistic use case than VM)
Recommend we get access to mCCF first, as this ticket can provide more value once we have an idea of ARM templates/mCCF world
Create E2E API Test. Test Data Ingest API.
Reference how e2e tests look in forum app with fake auth
https://github.com/microsoft/CCF/blob/7455dd869df4996d267af228d17a484a33cd67c7/samples/apps/forum/test/e2e/api.test.ts
Dependent on #82
ADR to define a generalised ingest API
Dependent on the LEI schema investigation
Currently there are certificates shipped with the repository for the demo in banking_app/certs_for_demo
. These could be created when the demo is initiated.
Design Data Reconciliation
Acceptance Criteria:
Dependent on conversation with Brent
When the CI is triggered it should build the devcontainer and use that to run the tests - this way we know that the devcontainer always works
Context from PG/Amaury:
The banking-app-sample will have its own constitution. Update the auditable-logging-app sample to follow the same pattern
Leverage the generic governance scripts for our skeleton app.
For the skeleton app, we can assume 1 member & 1 user for now...
Suggested by @heidihoward .
It was originally in CCF/samples/apps/forum at 7455dd869df4996d267af228d17a484a33cd67c7 · microsoft/CCF (github.com).
Maybe it doesn't highest priority, but it's worth to consider.
After cloning the repository, there should be a one line command to build the sample application(s)z
Context from PG/Amaury:
The https://github.com/microsoft/ccf-app-samples/ repository has previously had certificates stored inside it for the demo. These were replaced by dynamically creating the certificates as part of the demo.
However this means the certs are in the git history and this means it is problematic to fork this repository to Azure DevOps.
I think we should squash the commits before and after the certs being added now before we iterate further.
Define repository structure for /data-reconciliation-app in ccf-app-samples repo. Be sure to reference the banking-app and follow a similar structure for our app.
App structure
Reference template: https://github.com/microsoft/ccf-app-template/tree/main/js
CCF Stuff
Docs
Acceptance Criteria:
GitHub Actions should be enough.
Document get started repository for building CCF applications using (JavaScript and C++).
Steps to build network, open network, activate members, deploy app, etc.
"You would skip these steps if you use mCCF"
Next Steps (Nov 9th):
I will create a PR on the samples repo, and after revision and acceptance we can extract the generic parts and push it to the main CCF docs.
In the samples repo we have a config
and docker
folder in the root. These should be in each sample so that each sample can move at its own release cadence.
Design Data Reporting - API, Service & Repository Layers.
Requirements: https://github.com/microsoft/ccf-app-samples/blob/main/data-reconciliation-app/docs/data-schema-data-flow.md#data-reporting
Members will query for report summary. All data returned will be JSON. Voting Threshold should be configurable.
Design APIs:
Design Service Layer
Repository Layer:
Acceptance Criteria:
There may be another place in CCF that creates a set_app_js governance proposal document for us to be able to register an application in the network. So this may be unnecessary, so happy to remove this. As mentioned here, https://microsoft.github.io/CCF/main/build_apps/js_app_bundle.html#deployment we need to create set_js_app proposal.
If this is worthwhile addition I can also amend the documentation in the main CCF repository
Feature labels
Bug labels
Check in with MK so that we can slice a view by Feature/Story/Task
Per demo requirements, we will need to demonstrate governance actions = a code change.
https://github.com/microsoft/ccf-app-samples/blob/main/data-reconciliation-app/demo/demo-guidance.md
[SPIKE] - Proof that version 1 of the application is still running whilst there is a active proposal (version 2)
Create a script to
At the moment, running ./banking-app/test.sh only works the first time as it relies on a bank account balance which always increases.
Update the test so that you can run it multiple times.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.