In some circumstances the conduit proxy app might remain running forever.

For instance, if the cf conduit fails, or if the cf token expires or the user targets a different API endpoint or org or space.

If the application is running forever, it might happen that the user is billied for ever.

Proposed solution

One solution can run some kind of watchdog command in the app, that would fail if no SSH tunnel is created or if it stops. For example, dropping a safe-terminate.sh

#!/bin/bash

ssh_running() {
  netstat -t | grep -q ":2222"
}

start_timeout=$(date -d "+1 min" +%s)
max_timeout=$(date -d "+7 days" +%s)

while ! ssh_running && [ "$(date +%s)" -lt "${start_timeout}" ] ; do
  echo "Waiting for SSH session..."
  sleep 5
done

while ssh_running && [ "$(date +%s)" -lt "${max_timeout}" ] ; do
  echo "SSH session still running..."
  sleep 600
done

echo "No more SSH sessions or timeout detected. Exitting in 10 seconds."
sleep 10   

then, cf conduit would push the static app as follows:

cf push -b staticfile_buildpack -m 64m -k 64m -i 1 --health-check-type none --no-route --no-manifest __conduit_12345__  -c 'bash $HOME/public/safe-terminate.sh'

The app would wait for a SSH to be present in 1 minute, and then wait until it is gone. Then it will terminate after 10 seconds. In normal operation, the flow would be as usual, the application would be deleted once conduit finished.

If instead the cf conduit fails to delete the app, the SSH tunnel would eventually die. The script would detect that and terminate. CF would restart the application several times, but the script would fail after one minute because no SSH connection is done.

Hi,
I only need to setup postgres backups about once a year, even less often, which is infrequent enough to hit the same issues.
It would be good if there was a TROUBLESHOOTING section in the README, and ideal if there was also something cf diagnose-postgres and cf diagnose-mysql to find common connection issues.

• local version of postgres doesn't match remote version
  Currently, I just hit an error like this:

OK Preparing command: pg_dump -f backup.sql
pg_dump: error: connection to database "XXX-YYY-ZZZ" failed: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

It would be nice if there was a hint to run a diagnose command on the end, like this:

Postgres connection failed, you can try diagnosing connection issues with:
    cf conduit diagnose-postgres

Where cf conduit diagnose-postgres would check versions match and other connectivity issues (like whether the database port appears to be open, which can be achieved without even using psql)

Cheers
Stuart

What

We should detect service types (postgres/mysql) based on other heuristics (such as the connection uri string), and be more flexible about detecting and extracting credentials from the binding data.

Why

Currently we rely on the service name itself being "postgres" or "mysql" and binding credentials being in the same specific format as our rds-broker output.

This is great for us , but there's no reason why this plugin couldn't work for services provided by a different broker with different service names and slightly different binding data.

If we don't do this

The plugin will be less useful for the wider community

Steps to reproduce

• Start a conduit (to psql)
• Leave it open for a day
• Exit psql

What is expected

Exits cleanly and stops/deletes the __conduit__ app

What is observed

conduit app left in running state.

hypothesis

The access_token is expiring so the plugin is no longer authorized to clean up

This repo doesn't currently have any tests. That's OK as a proof of concept.

If users find it useful and depend on it then we should write tests and have them automatically run against pull requests so that it's easier to review changes with increased confidence.

Publish to https://plugins.cloudfoundry.org/ when we're happy user feedback and quality.

What

Delete the __conduit_XXXXX__ application and service binding when the utility exits.

If necessary, change the instructions to use ^D (close STDIN) rather than ^C (send SIGTERM).

Why

So that the application doesn't use resource quotas and the service credentials are revoked.

Hi

I've managed to get myself in a bit of a situation and I was hoping someone here might be able to help.

While testing paas, I have created a cf postgres service called pg-svc. I have then used the conduit plugin to connect to the instance and run psql. During the session I have then made some changes to the db as the (I believe) temporary pg role created by the conduit plugin.

Somehow though I have managed to disconnect the conduit session but the related conduit app (conduit_bupby9jm) has not been removed, it's still there in cf apps and on the gov.paas dashboard. The temp conduit user role is also still in the pg db.

I now can't remove the conduit app because Service broker error: pq: role "ugzehw_usjxjqmqt" cannot be dropped because some objects depend on it. Neither can I remove the pg service because Cannot delete service instance. Service keys, bindings, and shares must first be deleted.

If I try and unbind the service with cf unbind-service __conduit_bupby9jm__ pg-svc I get the same n unbind operation for the service binding between app __conduit_bupby9jm__ and service instance xws-db failed: Service broker error: pq: role "ugzehw_usjxjqmqt" cannot be dropped because some objects depend on it

I'm not sure what the pg objects are that depend on the temp conduit role as I don't fully recall what scripts I ran but querying the db doesn't seem to report any objects actually being owned by that role. I have tried to REASSIGN OWNED BY __conduit_bupby9jm__ TO anotherrole; but I don't have sufficient permissions and when I set the db up, I didn't specify CREATEROLE so I'm unable to give myself permission.

I hope that makes sense to someone. I can't even delete the space because of the above issues. Shy of trying to fully delete the org, I'm out of ideas. Any help would really be appreciated - thanks.

Hello,

I am installing Conduit for the first time but keep getting a Proxy Authentication error. I am using a Windows pc in our corporate environment. I have proxy settings set in my environment variables that allow me to use PaaS and Cloud Foundry okay however I cannot install this plugin. Is there any other alternatives for this or any ideas to help me? Thanks...

Searching CF-Community for plugin conduit...
Get https://plugins.cloudfoundry.org/list: Proxy Authentication Required
FAILED

I tried to connect to a db instance as normal this morning but I encountered this error. Screenshot included for further information. I was able to connect to this instance yesterday afternoon. I've tried logging out then logging in again and re-installing the conduit plugin. Unable to connect to mysql shell instance either.

I'm using conduit to supply VCAP_SERVICES to a bash script which does some processing. If the bash script fails conduit is swallowing the exit code.

Conduit should return the exit coder from my subcommand so that I can perform additional actions if required.

Here's a simple examples:

exit.sh

#!/bin/bash
exit 1

Normal operation:

$ ./exit.sh
$ echo $?
1

Wrapped by conduit:

$ cf7 conduit db -- ./exit.sh
OK Connecting client
...
OK Waiting for conduit app to become available
OK Fetching service infomation
OK Binding db
OK Fetching environment
OK Starting port forwarding
OK Waiting for port forwarding
OK Preparing command: ./exit.sh
$ echo $?
0

After a while conduit starts to write the following error messages:

Authorization server did not redirect with one time code
Authorization server did not redirect with one time code
[...]

I suspect the token needs to expire for this to happen.

I connected without an app:

cf conduit <service name>

Just want to note down one of the causes of conduit not closing down.

Hi,

I am developing an application on the GOV.UK PaaS for BEIS.

I have a GOV.UK PaaS account, and I have created a database with the following commands:

cf login -a api.cloud.service.gov.uk -u *** -p ***
cf target -o ***
cf create-space int
cf target -o *** -s "int"
cf create-service mysql Free myapp-database 
# wait until the database service is ready
cf install-plugin conduit

When I attempt to connect to the database with the following command:

cf conduit myapp-database -- mysql

I get the following error:

ERROR 1045 (28000): Access denied for user 'un73dmsmf7wo7anm'@'10.0.34.10' (using password: YES)

Does anyone know how I can diagnose or fix this issue?

Many thanks,

Rich

If neither the client or the server send any packets for a few minutes the server seems to hang up.

We should implement some kind of keepalive so that you can use conduit for things like long-running sql queries, or interactive sessions where you're not doing anything for a while.

Go's ssh client doesn't have a magic keepalive flag, but it does have a SendRequest() function which looks like it can achieve the same thing:

• https://godoc.org/golang.org/x/crypto/ssh#Session.SendRequest
• golang/go#5875 (comment)
• https://stackoverflow.com/a/31566330/1344760

When downloading the latest version of cf:
cf install-plugin conduit

I'm receiving version 0.0.15 instead of the expected version 0.1.0

When running one of the example scripts:
cf conduit <INSTANCE> -- pgdump -f backup.sql

I receive this error:
Error: Unknown program pgdump: can't determine what service types it expects

This happens for docker also when using the docker example.

A team mate that had this previously working (on version 0.0.12) is also experiencing the same issue after upgrading.

Temporary solution found by my team mate:
cf install-plugin https://github.com/alphagov/paas-cf-conduit/releases/download/v0.0.14/cf-conduit.darwin.amd64 -f

Steps to reproduce

Start a conduit to a shell/repl type command:

cf conduit mything -- psql

Send ctrl+c.

Expected behaviour

psql handles the ctrl+c as it sees fit ... in this case "clear the current input"

Observed behaviour

Get booted out of the psql session and the conduit closes.

Potential fix

When running a foreground command, signals should be forwarded to the child command.

org govuk-notify, space production, conduit name __conduit_79086__

Timeline:

conduit created (cf conduit notify-db -- psql) at 10:23
used normally at 10:24:59
entered another query at 10:44:46 - no response
killed manually using ctrl+c at 10:48:

rdsbroker_**********=> select * from notifications where sent_by = 'mmg' limit 5;
^[[A^[[A^[[B^[[B^C
[leohemsted:~]$ Could not send cancel request: PQcancel() -- connect() failed: Connection refused
SSL SYSCALL error: EOF detected
The connection to the server was lost. Attempting reset: Failed.
!> \q

[leohemsted:~]$

10:49 conduit still visible in cf apps

As part of the work to integrate the https://github.com/alphagov/paas-aiven-broker with the GOV.UK PaaS, we're documenting the user-facing ways of interacting with the services it provisions. cf conduit appears to fail to provide a tunnelled-via-an-app route to these services, instead displaying the backend host which is trivially available via cf env.

This is a problem because it doesn't work with services which have an IP whitelist in front of them and which allow only PaaS-originated traffic to reach the service instances.

In Aiven's case, this happens for the Elasticsearch service. It's exposed as an HTTPS URI, with embedded/inline credentials. Perhaps this URI is confusing conduit?

$ cf create-service elasticsearch small-ha-6.x es
Creating service instance foo in org govuk-paas / space demo as admin...
OK

Create in progress. Use 'cf services' or 'cf service es' to check operation status.

$ until cf services | grep ^es | grep -v 'create in progress'; do sleep 5 ; done ; cf conduit es
OK Connecting client
OK Targeting org govuk-paas
OK Targeting space demo
OK Deploying __conduit_8664__
OK Uploading __conduit_8664__ bits
OK Starting __conduit_8664__
OK Waiting for conduit app to become available
OK Fetching service infomation
OK Binding es
OK Fetching environment
OK Starting port forwarding
OK Waiting for port forwarding

The following services are ready for you to connect to:

* service: es (elasticsearch)
  hostname: redacted-1840a56-a302-a57eeb2d3090.aivencloud.com
  password: redacted
  port: 10264
  uri: https://786b50b1-20a0-4d06-b50b-96ca048e4b89:[email protected]:10264
  username: 786b50b1-20a0-4d06-b50b-96ca048e4b89

Press Ctrl+C to shutdown.

What

Any existing environment variables are not overridden when setting up the env for executing commands.

Steps to reproduce

$ PGDATABASE=xxx cf conduit posty -- env | grep PGDATABASE
PGDATABASE=xxx
PGDATABASE=rdsbroker_8xxxxx_2xxxxx_bxxx_bxxxxxxxxx

What is expected

Only the single (relevant) PGDATABASE variable in the environment.

Notes

Overwrite existing environment variables before exec.

Hello! I'm at @18F, and one of the authors of Service Connection Plugin. I was just going to submit the plugin to the Cloud Foundry Plugin Directory, and came across yours. It seems like the two projects have the same goal - any interest in combining forces on them?

Conduit allows running a command. For instance if connecting to a Postgres database you can directly run psql against it by cf conduit your-postgres -- psql. But Conduit does not warn you if running psql (or the equivalent for other services) against the wrong type of service. This has confused PaaS support and is reasonably likely to confuse other people.