opencast / opencast-docker Goto Github PK

View Code? Open in Web Editor NEW

41.0 41.0 36.0 1.1 MB

Dockerfiles for Opencast

Home Page: https://quay.io/organization/opencast/

License: Educational Community License v2.0

Shell 73.74% Makefile 6.40% Java 5.23% Dockerfile 14.63%

docker-images hacktoberfest opencast video-processing

opencast-docker's Introduction

Opencast

Open Source Lecture Capture & Video Management for Education

Opencast is a free, open-source platform to support the management of educational audio and video content. Institutions can use Opencast to produce lecture recordings, manage existing video, serve designated distribution channels, and provide user interfaces to engage students with educational videos.

Installation

Installation instructions can be found locally at docs/guides/admin/docs or in our online documentation.

Community

More information about the community:

opencast-docker's People

Contributors

Stargazers

Watchers

opencast-docker's Issues

Let user choose uid/gid of the Opencast build user for the build image

Currently the build image builds Opencast as root. This leads to problems if files are shared via volumes with a local Docker user. Instead when starting a build container, users should be able to specify a uid/gid of a new user within the container. This new user must be able to become root again, as the Opencast startup scripts require.

Create images for 2.3.3

Broken build caused by alpine edge package

At the moment it is not possible to build the docker images for the opencast version 6.2.
The reason is that the alpine edge repository does not contain the version 62 of libicui18n but version 63.

ERROR: unsatisfiable constraints:
so:libicui18n.so.62 (missing):
required by: tesseract-ocr-4.0.0-ro[so:libicui18n.so.62]
...

I had a similar problem with the images of oc version 5.

Isn't it better to use the stable alpine version?

Evaluate Tesseract 4.0 and the different models

See:

Publishing issue

I'm currently having issues publishing any videos that I process.
The initial processing before hold has no issues.
I'm using "Docker for Mac", could it be s factor?
The logs i'm getting are next to useless. just complaining about a Eclipse missing.

Create images for Opencast 3.1

Upgrade to Alpine 3.8

Add Opencast 3.7

See https://github.com/opencast/opencast/releases/tag/3.7

Add HEALTHCHECK

Since Docker 1.12 there are some new commands for Dockerfiles. HEALTHCHECK let you define a command for testing if your application is healthy. It is planed to use this in the Swarm load balancer in Docker 1.13.

Create images for 2.3.4

Is the 'next' tag at quay.io rebuild periodically?

Is the 'next' tag rebuild periodically?
Can it be rebuild every night?

Add Helm chart

Add Helm chart for Opencast.

missing manifest for opencast/<distribution>4.0

example:

Pulling opencast (opencast/allinone:4.0)...
ERROR: manifest for opencast/allinone:4.0 not found

By looking at the DockerHub the only 4.x tag is 4.4, while the only official 4.x version of Opencast is 4.0 and this also reflects in the docker-compose versioning.

Distributed Docker compose with MariaDB nodes can't communicate each others

I just tested this version and I have some issues:

The worker, presenter and the and the admin can't see each other because the URLs are configured without a proper alias or the internal docker network.

Create images for Opencast 3.5

Rename HSQL to H2

Enable cache support with Circle CI

... which would hopefully increase build time a lot.

docker secrets?

Is it posible tu use docker secrets instead of environment params for sensitive data?

Docker reports Opencast images as Unhealthy

Docker reports the opencast images as unhealthy, using the command

$ docker inspect --format='{{json .State.Health}}' <container-name>

This command shows that in all opencast nodes the problem comes from this line:

/docker-healthcheck.sh: 17: set: Illegal option -o pipefail\n

CAS support?

Hi!

I'm trying to enable CAS, following the steps from https://docs.opencast.org/latest/admin/configuration/security.cas/, but I'm stuck on the first step where I need to access to the karaf console to install a feature (feature:install opencast-contrib-cas).

Is there any way to access to the karaf console in these opencast images?

Add tag for next Opencast version

In order to ease the testing of new features it would be nice to have a version targeting the next major release of Opencast.

OC commands fail on 7.x branch

On 7.x branch files have been remove but are still required by the oc_* commands, therefore the following commands will fail:

bash-4.4$ oc_install
==> [oc_install] No distribution given as first argument. Using 'develop'.
==> [oc_install] Start installation
==> [oc_install] Copy develop
==> [oc_install] Create folders
==> [oc_install] Copy Docker scripts
==> [oc_install] Copy support files
==> [oc_install] Copy configuration
cp: cannot stat '/docker/etc/develop/index/adminui/settings.yml': No such file or directory
bash-4.4$ oc_run
==> [oc_run] Start run
sh: /opencast/docker/scripts/env: No such file or directory

Fix unnecessary large image size because of late chown

Evaluate MH-12023 for the build image

Currently the build image includes some helper scripts that build all Opencast distributions. MH-12023 introduced a profile for only building the allinone distribution. To speed up the build process, maybe the scripts should only assemble the allinone distribution by default.

Create images for Opencast 4.1

ORG_OPENCASTPROJECT_DB_JDBC_URL and useSSL parameter

I've been having trouble with passing the useSSL=false parameter to the JDBC connection string.
E.g.
docker run -e "ORG_OPENCASTPROJECT_DB_VENDOR=MySQL" -e "ORG_OPENCASTPROJECT_DB_JDBC_URL=jdbc:mysql://oc-mysql:3306/opencast?useSSL=false" ..other configs...
produces

Run opencast_jdbc_trytoconnect
Try to connect to DB (1/25) Mon Nov 12 12:43:28 UTC 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

It seems to be the case that the useSSL=false parameter is not appended to the JDBC connection string.

Looking at Dockerfiles/{distribution}/assets/scripts/jdbc.sh, the connection string is parsed with

url=$(awk -F "=" '/org.opencastproject.db.jdbc.url/ {print $2}' etc/custom.properties | tr -d ' ')

Using = as a delimiter, the second column for

ORG_OPENCASTPROJECT_DB_JDBC_URL=jdbc:mysql://oc-mysql:3306/opencast?useSSL=false

results in

jdbc:mysql://oc-mysql:3306/opencast?useSSL.

Instead of only being the bearer of bad news, I come bearing gifts :)
An alternative awk command like this should work:

url=$(awk -F "=" '/org.opencastproject.db.jdbc.url/ {url = ""; for (i=2; i <=NF; i++) url = url$i"="; print url}' etc/custom.properties | sed 's/=$//' | tr -d ' ')

The above command (or whatever ends up being accepted) should be applied to ACTIVEMQ_BROKER_URL as that connection string also allows the passing of parameters.

Evaluate MH-11988: Change karaf Shutdown Command

With MH-11988 Opencast 2.3.1 now advises to change the karaf shutdown command. I would argue that this is not necessary for these Docker images as anyone, who has access to the shutdown port, is already within the container, probably as root. But I would like to here other opinions.

cc @JanKoppe @lkiesow

Change example ActiveMQ image

The ActiveMQ image webcenter/activemq used in the examples is outdated (especially the base image). There are some more recent commits in the develop branch, but nothing has been published to Docker Hub yet.

Add Opencast 6.0

Provide presentation-apache Image with stream security plugin?

I think it would be a good idea to provide an additional presentation-apache image that would include a Apache HTTP Server as a reverse proxy with the Stream Security plugin included.

UnknownHostException with docker-compose.allinone.hsql

Tried docker-compose.allinone.hsql.yml on Mac using the command line from the README.

OC was not able to register services at its service registry. Cause was an UnknownHostException saying 'Unknown host opencast'.

Changing the host to localhost:8080 in line 30 solved the issue for me.

I'm new to docker so maybe I just did not configure my machine right. Wanted to report though.

Make ORG_OPENCASTPROJECT_SERVER_URL optional by using the hostname of the container

These images enable Opencast to be run in a Docker orchestrator environment (e.g. Docker Swarm). Most of these environments have scaling functionality build in (useful e.g. for Opencast worker). Opencast requires that each instance has a unique server URL, which, at the moment, is set by the ORG_OPENCASTPROJECT_SERVER_URL environment variable. Orchestrator that cannot alter this value for each instance (e.g. Docker until version 1.12) cannot scale Opencast instances. To support these

ORG_OPENCASTPROJECT_SERVER_URL should be optional
if ORG_OPENCASTPROJECT_SERVER_URL is not set "http://<hostname>:8080" should be used

Using "http://<ip address>:8080" would be an alternative, but within the container it is not clear what interface is the correct one to use (i.e. the overlay network). Simply using the hostname works with

Docker Swarm mode
...

Other orchestrator might need other values. Support should be added if needed.

Update health-check scripts for new endpoint

Update health-check scripts for the new endpoint in Opencast 8.

Default Opencast DB pool max.idle.time bigger than MariaDB wait_timeout (1h vs 10m)

Hi!

Because of the Debian package used in MariaDB's docker image, the default value of wait_timeout isn't 28800 (8h), but 600 (10m) - which is below Opencast's default for org.opencastproject.db.jdbc.pool.max.idle.time, 3600 (1h)

This results in Opencast using closed connections without reopening them correctly (maybe another issue to solve)

2019-04-16 08:58:46,735 | WARN  | (Slf4jMLog$Slf4jMLogger$WarnLogger:220) - [c3p0] A PooledConnection that has already signalled a Connection error is still in use!
2019-04-16 08:58:46,736 | WARN  | (Slf4jMLog$Slf4jMLogger$WarnLogger:223) - [c3p0] Another error has occurred [ com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed. ] which will not be reported to listeners!
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.8.0_212]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)[:1.8.0_212]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.8.0_212]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)[:1.8.0_212]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1187)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.Util.getInstance(Util.java:408)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1182)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4068)[69:opencast-db:6.4.0]
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:567)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4037)[69:opencast-db:6.4.0]
	at org.eclipse.persistence.internal.databaseaccess.DatabasePlatform.wasFailureCommunicationBased(DatabasePlatform.java:2917)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1616)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:676)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.platform.server.ServerPlatformBase.wasFailureCommunicationBased(ServerPlatformBase.java:546)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:570)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2056)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:560)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectAllRows(DatasourceCallQueryMechanism.java:694)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRowsFromTable(ExpressionQueryMechanism.java:2740)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:559)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:258)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:242)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:299)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRows(ExpressionQueryMechanism.java:2693)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1175)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1222)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:904)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:460)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1134)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2896)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.retryQuery(UnitOfWorkImpl.java:5536)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.retryQuery(AbstractSession.java:1928)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1857)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.retryQuery(UnitOfWorkImpl.java:5536)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1894)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.sessions.server.ClientSession.retryQuery(ClientSession.java:694)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.sessions.server.ClientSession.retryQuery(ClientSession.java:694)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.retryQuery(AbstractSession.java:1928)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1894)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:258)[324:org.eclipse.persistence.jpa:2.6.4.v20160829-44060b6]
	at org.opencastproject.serviceregistry.impl.ServiceRegistryJpaImpl$JobDispatcher.run(ServiceRegistryJpaImpl.java:2934)[149:opencast-serviceregistry:6.4.0]
	at org.opencastproject.serviceregistry.impl.ServiceRegistryJpaImpl.getDispatchableJobsWithStatus(ServiceRegistryJpaImpl.java:1878)[149:opencast-serviceregistry:6.4.0]
	at org.eclipse.persistence.internal.jpa.QueryImpl.getResultList(QueryImpl.java:473)[324:org.eclipse.persistence.jpa:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1839)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)[:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_212]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_212]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)[:1.8.0_212]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)[:1.8.0_212]
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
	at java.lang.Thread.run(Thread.java:748)[:1.8.0_212]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)[:1.8.0_212]
The last packet successfully received from the server was 10,008 milliseconds ago.  The last packet sent successfully to the server was 0 milliseconds ago.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.8.0_212]
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)[69:opencast-db:6.4.0]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)[:1.8.0_212]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.8.0_212]
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3459)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3900)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2494)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3559)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858)[69:opencast-db:6.4.0]
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:353)[69:opencast-db:6.4.0]
	at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1966)[69:opencast-db:6.4.0]
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:644)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:1009)[323:org.eclipse.persistence.core:2.6.4.v20160829-44060b6]
	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3011)[69:opencast-db:6.4.0]
	... 39 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3469)[69:opencast-db:6.4.0]
	... 49 more

Neither am I sure why Docker's default seems to be 600, nor which default timeout should be changed.
I'll try to increase wait_timeout for my environment using

command: [
  '--wait_timeout=28800'
]

in docker-compose.yml.

Update configuration for Opencast 2.2.1

Some configuration changes were not adopted.

Use LABEL instead of deprecated MAINTAINER

Docker 1.13 deprecated the MAINTAINER command.

Swarm Example

Not Really an Issue. More of a question.

Have you guys tried the new Swarm capabilities of Docker with opencast?
I'm easily overpowering my test iMac with opencast's requirements and would like to scale it out to my other test iMacs. I could use the opencast model and will eventually use it for sure, but I was wondering how the new Swarm capabilities built into Docker would work out.

Any examples?

Worker on another Node Not talking to ActiveMQ

I'm still mucking around with "Docker for Mac".
Just finally learned today that Docker Swarm doesn't work on "Docker for Mac" outside on external nodes! Well thats useless.

So tried to just use my other iMacs as workers. Now I'm stuck on the activeMQ not making a connection to the Docker instance on the master Node.

I wish I had more info, but I had to abandon this project as a class was going to use my make shift nodes. Any Advice on what to look for the next time I try this? How do I test that the ActiveMQ is responding on external ports? It's the only thing I think; "besides misconfigured settings"; that could be causing it to hang. I know nothing about ActiveMQ. Wish I could remove it's function to test the other stuff first.

Create Images for 2.2.5

Add an example for Docker Swarm mode

Change default behaviour of build scripts to use the develop distribution

This is only relevant for the build image. Currently one has to explicitly ask for the develop distribution to be build. Since this is probably the most used distribution during development this should be the default. 4.0 is probably the best point to introduce this change.

Remove Opencast PID file once container process is stopped

Sometimes Opencast is terminated and its PID file is not cleaned up. Upon restarting the container Opencast rejects to start. It should be ensured that the PID file is removed once Opencast terminates.

NoRouteToHostException: Host is unreachable in Docker

Hey guys,

I am using docker-compose 1.19, in a CentOS 7 system.

After start docker like this,

$ export HOSTIP=10.100.176.131
$ docker-compose -f docker-compose/docker-compose.allinone.h2.yml up
The website of opencast is fine, after upload a mp4 file, I got a error here:

opencast_1  | 2018-02-14 16:55:34,245 | INFO  | (FFmpegSilenceDetector:148) - Track /data/opencast/workspace/mediapackage/246b655f-d6d3-4f6f-804c-9fa793b938f7/a4a842d0-15c4-44bb-b498-05f5ad346b30/__________.mp4 loaded, duration is 60 s
opencast_1  | 2018-02-14 16:55:34,246 | INFO  | (FFmpegSilenceDetector:150) - Starting silence detection of /data/opencast/workspace/mediapackage/246b655f-d6d3-4f6f-804c-9fa793b938f7/a4a842d0-15c4-44bb-b498-05f5ad346b30/__________.mp4
opencast_1  | 2018-02-14 16:55:34,246 | INFO  | (FFmpegSilenceDetector:158) - Running ffmpeg -nostats -i /data/opencast/workspace/mediapackage/246b655f-d6d3-4f6f-804c-9fa793b938f7/a4a842d0-15c4-44bb-b498-05f5ad346b30/__________.mp4 -filter:a silencedetect=noise=-40dB:duration=10.000 -f null -
opencast_1  | 2018-02-14 16:55:34,540 | INFO  | (FFmpegSilenceDetector:190) - No silence found. Adding one large segment.
opencast_1  | 2018-02-14 16:55:34,542 | INFO  | (FFmpegSilenceDetector:238) - Segmentation of track a4a842d0-15c4-44bb-b498-05f5ad346b30 yielded 1 segments
opencast_1  | 2018-02-14 16:55:39,291 | INFO  | (SilenceDetectionWorkflowOperationHandler:216) - Finished silence detection on track a4a842d0-15c4-44bb-b498-05f5ad346b30
opencast_1  | 2018-02-14 16:55:44,313 | INFO  | (TagWorkflowOperationHandler:122) - Retagging mediapackage elements
opencast_1  | 2018-02-14 16:55:49,368 | INFO  | (ConfigurablePublishWorkflowOperationHandler:289) - Start bulk publishing of 2 elements of media package '246b655f-d6d3-4f6f-804c-9fa793b938f7' to publication channel 'internal'
opencast_1  | 2018-02-14 16:55:58,333 | WARN  | (DownloadDistributionServiceImpl:359) - Error distributing http://opencast:8080/files/mediapackage/246b655f-d6d3-4f6f-804c-9fa793b938f7/44734aaf-e614-4c52-874a-d096ad411655/waveform.png
opencast_1  | org.opencastproject.security.api.TrustedHttpClientException: java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
opencast_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:392)[106:matterhorn-kernel:4.1.0]
opencast_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:345)[106:matterhorn-kernel:4.1.0]
opencast_1  | 	at org.opencastproject.security.util.StandAloneTrustedHttpClientImpl$1.apply(StandAloneTrustedHttpClientImpl.java:128)[55:matterhorn-common:4.1.0]
opencast_1  | 	at org.opencastproject.security.util.StandAloneTrustedHttpClientImpl$1.apply(StandAloneTrustedHttpClientImpl.java:123)[55:matterhorn-common:4.1.0]
opencast_1  | 	at org.opencastproject.util.HttpUtil.waitForResource(HttpUtil.java:208)[55:matterhorn-common:4.1.0]
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.distributeElement(DownloadDistributionServiceImpl.java:346)[73:matterhorn-distribution-service-download:4.1.0]
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.distributeElements(DownloadDistributionServiceImpl.java:244)[73:matterhorn-distribution-service-download:4.1.0]
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.process(DownloadDistributionServiceImpl.java:501)[73:matterhorn-distribution-service-download:4.1.0]
opencast_1  | 	at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:282)[55:matterhorn-common:4.1.0]
opencast_1  | 	at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:241)[55:matterhorn-common:4.1.0]
opencast_1  | 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_151]
opencast_1  | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_151]
opencast_1  | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_151]
opencast_1  | 	at java.lang.Thread.run(Thread.java:748)[:1.8.0_151]
opencast_1  | Caused by: java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
opencast_1  | 	at java.net.PlainSocketImpl.socketConnect(Native Method)[:1.8.0_151]
opencast_1  | 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)[:1.8.0_151]
opencast_1  | 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)[:1.8.0_151]
opencast_1  | 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)[:1.8.0_151]
opencast_1  | 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)[:1.8.0_151]
opencast_1  | 	at java.net.Socket.connect(Socket.java:589)[:1.8.0_151]
opencast_1  | 	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)[236:org.apache.httpcomponents.httpclient:4.5.0]
opencast_1  | 	at org.opencastproject.kernel.http.impl.HttpClientImpl.execute(HttpClientImpl.java:77)[106:matterhorn-kernel:4.1.0]
opencast_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:385)[106:matterhorn-kernel:4.1.0]
opencast_1  | 	... 13 more
opencast_1  | 2018-02-14 16:55:58,340 | INFO  | (ServiceRegistryJpaImpl:2436) - State set to WARNING for current service org.opencastproject.distribution.download on host http://opencast:8080
opencast_1  | 2018-02-14 16:55:58,355 | ERROR | (AbstractJobProducer$JobRunner:312) - Error handling operation 'Distribute': org.opencastproject.serviceregistry.api.ServiceRegistryException: Error handling operation 'Distribute'
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.process(DownloadDistributionServiceImpl.java:517)
opencast_1  | 	at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:282)
opencast_1  | 	at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:241)
opencast_1  | 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
opencast_1  | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
opencast_1  | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
opencast_1  | 	at java.lang.Thread.run(Thread.java:748)
opencast_1  | Caused by: org.opencastproject.distribution.api.DistributionException: org.opencastproject.security.api.TrustedHttpClientException: java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.distributeElement(DownloadDistributionServiceImpl.java:363)
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.distributeElements(DownloadDistributionServiceImpl.java:244)
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.process(DownloadDistributionServiceImpl.java:501)
opencast_1  | 	... 6 more
opencast_1  | Caused by: org.opencastproject.security.api.TrustedHttpClientException: java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
opencast_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:392)
opencast_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:345)
opencast_1  | 	at org.opencastproject.security.util.StandAloneTrustedHttpClientImpl$1.apply(StandAloneTrustedHttpClientImpl.java:128)
opencast_1  | 	at org.opencastproject.security.util.StandAloneTrustedHttpClientImpl$1.apply(StandAloneTrustedHttpClientImpl.java:123)
opencast_1  | 	at org.opencastproject.util.HttpUtil.waitForResource(HttpUtil.java:208)
opencast_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.distributeElement(DownloadDistributionServiceImpl.java:346)
opencast_1  | 	... 8 more
opencast_1  | Caused by: java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
opencast_1  | 	at java.net.PlainSocketImpl.socketConnect(Native Method)
opencast_1  | 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
opencast_1  | 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
opencast_1  | 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
opencast_1  | 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
opencast_1  | 	at java.net.Socket.connect(Socket.java:589)
opencast_1  | 	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
opencast_1  | 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
opencast_1  | 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
opencast_1  | 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
opencast_1  | 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
opencast_1  | 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
opencast_1  | 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
opencast_1  | 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
opencast_1  | 	at org.opencastproject.kernel.http.impl.HttpClientImpl.execute(HttpClientImpl.java:77)
opencast_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:385)
opencast_1  | 	... 13 more
opencast_1  | 
opencast_1  | 2018-02-14 16:55:59,376 | ERROR | (WorkflowOperationWorker:182) - Workflow operation 'operation:'publish-configure', position:21, state:'FAILED'' failed
opencast_1  | org.opencastproject.workflow.api.WorkflowOperationException: At least one of the distribution jobs did not complete successfully
opencast_1  | 	at org.opencastproject.workflow.handler.distribution.ConfigurablePublishWorkflowOperationHandler.distribute(ConfigurablePublishWorkflowOperationHandler.java:317)[74:matterhorn-distribution-workflowoperation:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.handler.distribution.ConfigurablePublishWorkflowOperationHandler.start(ConfigurablePublishWorkflowOperationHandler.java:236)[74:matterhorn-distribution-workflowoperation:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.impl.WorkflowOperationWorker.start(WorkflowOperationWorker.java:233)[185:matterhorn-workflow-service-impl:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.impl.WorkflowOperationWorker.execute(WorkflowOperationWorker.java:155)[185:matterhorn-workflow-service-impl:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.impl.WorkflowServiceImpl.runWorkflowOperation(WorkflowServiceImpl.java:885)[185:matterhorn-workflow-service-impl:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.impl.WorkflowServiceImpl.process(WorkflowServiceImpl.java:1895)[185:matterhorn-workflow-service-impl:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:2315)[185:matterhorn-workflow-service-impl:4.1.0]
opencast_1  | 	at org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:2281)[185:matterhorn-workflow-service-impl:4.1.0]
opencast_1  | 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_151]
opencast_1  | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_151]
opencast_1  | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_151]
opencast_1  | 	at java.lang.Thread.run(Thread.java:748)[:1.8.0_151]
opencast_1  | 2018-02-14 16:56:09,454 | INFO  | (AssetManagerSnapshotWorkflowOperationHandler:104) - Take snapshot of media package 246b655f-d6d3-4f6f-804c-9fa793b938f7
opencast_1  | 2018-02-14 16:56:09,460 | INFO  | (AbstractAssetManager:172) - Creating new version 1 of media package 246b655f-d6d3-4f6f-804c-9fa793b938f7
opencast_1  | 2018-02-14 16:56:09,472 | INFO  | (AssetManagerWithMessaging:207) - Send update message for snapshot 246b655f-d6d3-4f6f-804c-9fa793b938f7, 1 to ActiveMQ
opencast_1  | 2018-02-14 16:56:09,483 | INFO  | (WorkspaceImpl:392) - Downloading http://opencast:8080/assets/assets/246b655f-d6d3-4f6f-804c-9fa793b938f7/7eae3512-f4b6-49ff-b41f-157e348f57b1/1/dublincore.xml to /data/opencast/workspace/http_opencast_8080/assets/assets/246b655f-d6d3-4f6f-804c-9fa793b938f7/7eae3512-f4b6-49ff-b41f-157e348f57b1/1/dublincore.xml
opencast_1  | 2018-02-14 16:56:15,513 | INFO  | (WorkingFileRepositoryImpl:172) - [>a5336c92] Unable to delete non existing media package element 6df209b8-ca65-405d-8643-d19e6cb383e4@246b655f-d6d3-4f6f-804c-9fa793b938f7
^CGracefully stopping... (press Ctrl+C again to force)

Then the job failed, I got a error in Service tab.
which service name is org.opencastproject.distribution.download, and status warning.

So, is there any thing wrong with my setps? Thanks.

BTW, selinux and firewall all stopped in CentOS.

Publish failing when running multiserver docker compose

Encountering a failure to publish event when running Opencast with the provided multiserver docker compose (docker-compose.multiserver.mariadb.yml).

Workflow: Fast Testing Workflow
Distrubute to Opencast Media Module

Workflow fails trying to run the publish-configure operation.

opencast-presentation_1  | 2019-11-18T01:37:16,950 | WARN  | (DownloadDistributionServiceImpl:359) - Error distributing http://localhost:8080/files/mediapackage/09902b98-ab6d-4893-aacb-28db12405bb6/5adfe4ca-648c-4d1b-a6c6-d2fb14eda983/1mb.mp4
opencast-presentation_1  | org.opencastproject.security.api.TrustedHttpClientException: java.net.ConnectException: Connection refused (Connection refused)
opencast-presentation_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:394) ~[?:?]
opencast-presentation_1  | 	at org.opencastproject.kernel.security.TrustedHttpClientImpl.execute(TrustedHttpClientImpl.java:346) ~[?:?]
opencast-presentation_1  | 	at org.opencastproject.security.util.StandAloneTrustedHttpClientImpl$1.apply(StandAloneTrustedHttpClientImpl.java:128) ~[45:opencast-common:7.4.0]
opencast-presentation_1  | 	at org.opencastproject.security.util.StandAloneTrustedHttpClientImpl$1.apply(StandAloneTrustedHttpClientImpl.java:123) ~[45:opencast-common:7.4.0]
opencast-presentation_1  | 	at org.opencastproject.util.HttpUtil.waitForResource(HttpUtil.java:208) ~[45:opencast-common:7.4.0]
opencast-presentation_1  | 	at org.opencastproject.distribution.download.DownloadDistributionServiceImpl.distributeElement(DownloadDistributionServiceImpl.java:346) [52:opencast-distribution-service-download:7.4.0]

Upgrade Alpine base image

New 6.4 images does not have nc command

The new 6.4 images based on debian does not have the nc command needed by stop-opencast

[email protected]    | Run opencast_main_stop
[email protected]    | bin/stop-opencast: line 138: nc: command not found

Recent dockerhub builds fail because of --chown flag

E.g. https://hub.docker.com/r/opencast/allinone/builds/b79ez8ydjprypysjzjgmhd6/

this is for the 4.3 / latest release. I can't see an error in the dockerfile but FYI.

test

tmp

Add M2_REPO support for build image

Developers should be able to use a custom local Maven repository location.

Use Karaf Environment variables syntax instead of {{ENV_VAR}}

Karaf allow reference environment variables inside the configuration files using the syntax ${env:FOO}

From Karaf docs:

Environment variables can be referenced inside configuration files using the syntax ${env:<name>} (e.g. property=${env:FOO} will set "property" to the value of the enviroment variable "FOO").

This image uses a {{FOO}} and a sed command to replace de variables.

From custom.properfies:

org.opencastproject.server.url={{ORG_OPENCASTPROJECT_SERVER_URL}}

From helper.sh:

opencast_helper_replaceinfile() {
  file="$1"
  shift
  for var in "$@"; do
    eval exp_var="\$${var}"
    # shellcheck disable=SC2154
    sed -ri "s/[{]{2}${var}[}]{2}/$( echo "${exp_var}" | sed -e 's/[\/&]/\\&/g' )/g" "${file}"
  done
}

This does not allow me to mount a custom config file in read only mode or use a docker config in /opencast/etc/custom.properties.

I propose to replace the {{FOO}} by ${env:FOO} in Karaf config files and remove the sed commands.
What do you think?