-
An existing durable rabbitmq exchange (e.g.:
events
) that can be used to publish messages which get routed to the case service and fwmtadapter (e.g.:event.response.receipt
) -
A GCS bucket with a Cloud Pub/Sub notification configuration.
-
Relevant environment variables:
GOOGLE_APPLICATION_CREDENTIALS LOG_LEVEL RABBIT_HOST RABBIT_PORT RABBIT_VIRTUALHOST RABBIT_USERNAME RABBIT_PASSWORD RABBIT_EXCHANGE RABBIT_ROUTE RECEIPT_TOPIC_NAME RECEIPT_TOPIC_PROJECT_ID SUBSCRIPTION_NAME SUBSCRIPTION_PROJECT_ID READINESS_FILE_PATH
-
Pipenv for local development.
- Running the unit tests locally
- With an existing, accessible Google Cloud Project
- Without GCP (local emulation)
- Install dev packages:
pipenv install --dev
- Run the unit tests with pytest:
pipenv run pytest test/
Or:
make test
- Create a GCS bucket with a Cloud Pub/Sub notification configuration:
gsutil mb -c regional -l europe-west2 -p [TOPIC_PROJECT_ID] gs://[BUCKET_NAME]
gsutil notification create -t [TOPIC_NAME] -f json gs://[BUCKET_NAME]
gcloud beta pubsub subscriptions create --topic [TOPIC_NAME] [SUBSCRIPTION_NAME]
- Start RM services in Docker:
git clone [email protected]:ONSdigital/ras-rm-docker-dev.git
cd ras-rm-docker-dev && make up
- Create
.env
file in census-rm-pubsub directory:
cat > .env << EOS
RABBIT_HOST=localhost
RABBIT_PORT=6672
RABBIT_VIRTUALHOST=/
RABBIT_USERNAME=guest
RABBIT_PASSWORD=guest
SUBSCRIPTION_PROJECT_ID=[SUB_PROJECT_ID]
RECEIPT_TOPIC_PROJECT_ID=[TOPIC_PROJECT_ID]
GOOGLE_APPLICATION_CREDENTIALS=[/path/to/service/account/key.json]
RABBIT_EXCHANGE=events
RABBIT_ROUTE=event.response.receipt
RECEIPT_TOPIC_NAME=[TOPIC_NAME]
READINESS_FILE_PATH=/tmp/pubsub_ready
SUBSCRIPTION_NAME=[SUBSCRIPTION_NAME]
EOS
- Run the census-rm-pubsub application:
pipenv run python run.py
- Upload a file to the
gs://[BUCKET_NAME]
bucket, e.g.:
case_uuid=`python -c "import uuid;print(uuid.uuid4())"`
tx_uuid=`python -c "import uuid;print(uuid.uuid4())"`
touch $tx_uuid
gsutil -h x-goog-meta-case_id:$case_uuid \
-h x-goog-meta-tx_id:$tx_uuid \
cp $tx_uuid gs://[BUCKET_NAME]
- Start RM services in Docker:
git clone [email protected]:ONSdigital/ras-rm-docker-dev.git
cd ras-rm-docker-dev && make up
- Start Cloud Pub/Sub emulator:
gcloud components install pubsub-emulator
gcloud components update
gcloud beta emulators pubsub start
- Get Pub/Sub emulator-related environment variables:
gcloud beta emulators pubsub env-init
example output:
export PUBSUB_EMULATOR_HOST=::1:8410
- Create
.env
file in census-rm-pubsub directory:
cat > .env << EOS
RABBIT_HOST=localhost
RABBIT_PORT=6672
RABBIT_VIRTUALHOST=/
RABBIT_USERNAME=guest
RABBIT_PASSWORD=guest
SUBSCRIPTION_PROJECT_ID=project # can be anything
RECEIPT_TOPIC_PROJECT_ID=project # can be anything
PUBSUB_EMULATOR_HOST=localhost:8410 # taken from the env-init (above)
RABBIT_EXCHANGE=events
RABBIT_ROUTE=event.response.receipt
RECEIPT_TOPIC_NAME=eq-submission-topic
SUBSCRIPTION_NAME=rm-receipt-subscription
READINESS_FILE_PATH=/tmp/pubsub_ready
EOS
- Run the census-rm-pubsub application:
pipenv install
pipenv shell
python test/scripts/create_topic.py $RECEIPT_TOPIC_PROJECT_ID $RECEIPT_TOPIC_NAME
python test/scripts/create_subscription.py $RECEIPT_TOPIC_PROJECT_ID $RECEIPT_TOPIC_NAME $SUBSCRIPTION_NAME
python run.py
- In a separate terminal, tail the logs of the case service:
docker logs casesvc -f
- In a separate terminal, publish a message to the Pub/Sub emulator:
pipenv shell
python test/scripts/publish_message.py $RECEIPT_TOPIC_PROJECT_ID $RECEIPT_TOPIC_NAME
To run the component tests kill and remove any conflicting docker containers census-pubsub, pubsub & rabbitmq. If you have container clashes you will have to kill off other containers. To run the tests
make test
make down
This should successfully run the unit tests, build the latest census-rm-pubsub image, bring up rabbitmq, pubsub-emulator and census-rm-pubsub and then run the component tests on them. 'make down' will remove the containers if required.