How to start an application in remote docker containers using .NET Core

This article aims to show the process of starting an application in remote docker containers using .NET Core, to pass parameters to stdin, and receive an output from the console. To not add additional dependencies to the host machine, it had been decided to start the application in docker container on a remote server, and it was needed to organise an interaction between .NET Core and the docker. As the docker container and the .NET Core app are located on different servers, so it requires additional efforts to get it to work.

Launching applications in docker containers from .NET Core

So, this article shows you how to solve the task using the Docker.DotNet library.

The task of running a console application is typical, so, to not complicate it with business-specific requirements, the Linux sort command will be used as an example. Of course, instead of the sort command you can run any other console application written on Java or any other language.

So, let’s solve the task:

The sort command gets the list of lines via stdin, divided by the symbol of the line feed.

At the command line, run of sort looks like:

echo -e “3\n2\n1” | sort -n

We get the output:

1
2
3

To launch this command in the docker, you need to run the following command under the root user:

echo -e “3\n2\n1” | docker run --rm -i java:latest /bin/bash -c ‘sort -n’
1
2
3

Here are:

• --rm indicates that the container is to be deleted after the command execution
• -i shows to the docker that it is needed to get data from stdin
• java:latest is the image of the docker container, where we execute the command
• /bin/bash -c 'sort -n'is the command, which is run in the docker container

To run this command from .NET Core on the same machine, you can create the process of the docker through Process.Start, transfer all the required parameters to it and read OutputStream, transferred by the RedirectOutputStream parameter.

In this case, calling the command line is to be involved, the docker client is to be set on the host, and the docker should be launched under a privileged user.

As the alternative, you can use the Docker Remote REST API. It will allow you to run docker container not only on the application host but on any remote host over the internet. For integration with the Docker API in .NET Core, the library Docker.DotNet was used.

Let’s use this library. To start you should enable access to Remote API on the server, where the docker is installed. Then you should create certificates and enable Remote API in accordance with the documentation https://docs.docker.com/engine/security/https/ and https://success.docker.com/article/how-do-i-enable-the-remote-api-for-dockerd.

From files of client keys key.pem and cert.pem let’s create pfx file and download it to the machine, where you will build and launch our .NET Core project based on the Docker.DotNet library:

openssl pkcs12 -export -inkey key.pem -in cert.pem -out docker-key.pfx

Further, you should create .NET Core console project and add two NuGet packages: Docker.DotNet and Docker.DotNet.X509

As a next step, you should create a docker client in the code:

Because the server key is self-signed, you will disable the validation of the certificate through credentials.ServerCertificateValidationCallback. It is not safe, thus, to avoid connecting to the malicious server you can check that c.GetCertHashString() matches with a SHA1 hash of the certificate, created on the docker server instead of return always true.

After creation of the client, you can check that everything is working, and you can connect to the server where docker is located.

If you cannot connect you should check that port 2376 is opened on the server where docker is located. If you use clouds like AWS or Azure, you should add Inbound Rule for TCP 2376.

After you have checked that everything is working and you can connect to Docker API, let’s see the docker command should be implemented on .NET:

docker run — rm -i java:latest /bin/bash -c 'sort -n'

Unfortunately, in Docker API there are no ability to compile all command line parameters into one API request and this simple command line should be split into several API requests:

docker container create # create container
docker container attach # attaching stdin/stdout
docker container start # start container
docker container remove # remove container which completed task

Further let’s create the container:

It is very important to set properties OpenStdin and StdinOnce both to true. Without setting these properties the docker won’t read from stdin!

As the second step, the container is attached to the stdin, stdout and stderr streams:

This method returns MultiplexedStream which will be used later to write data for stdin and read return data from stdout.

Let’s write some data to stdin:

Please note that you have to call method CloseWrite(). After calling this method docker will know that all data is sent and the docker can close stdin.

Then let’s tell the docker that stdout should be saved into outputStream.

When these settings are done, you can start the docker container. It is not obvious that the method should be called, because it looks like writing to the stream of the container and only after that we should start the container. Please note that if you change the sequence of commands nothing will work!

Then you should wait when the program, which is run inside the docker container, finishes its output to stdout.

After that the container can be removed:

Finally, let’s get the results of the executed command from the memory stream:

And let’s get the output:

1
2
3

Conclusions

So, we have coped with the task! We have executed the command in docker container, which is located on remote server, sent the incoming data to stdin and read output data from stdout. Thus, we can launch any console application using this method not only standard linux commands.

Unfortunately, this method has a restriction in implementation of Docker.DotNet: if the console application does not regularly sends data to stdout and it waits for a while (for example, makes long-time computation and then it outputs results to the console), in this case Docker.DotNet breaks connection in 10 seconds and throws exception in the CopyOutputToAsync method. I was able to workaround this issue by rewriting CopyOutputToAsync method. If this article will be interested to readers and get fifty claps I will show how to do this in the next article.