Very High CPU Usage when running non-root containers

Platform API .NET SDK
1

We've discovered an issue when using the PureCloud API in an application running within a container started as a non-root user.

The configuration attempts to find/set a configuration file at user home by default: platform-client-sdk-dotnet/build/src/PureCloudPlatform.Client.V2/Client/Configuration.cs at 3c6ec88151e2cb607f7c0ac0ed23ec52636721bc · MyPureCloud/platform-client-sdk-dotnet · GitHub

then, it executes a config checker:

The config checker swallows the error which would indicate not having any permission to the default location.
(nothing in the catch, no re-throw, etc... to the application implementing the sdk).

Because of this, it doesn't appear anything is wrong and it causes excessive CPU usage which we were only able to find using an APM tool such as DataDog which shows CPU time spent on each individual method.

When we start the containers in kubernetes, we choose a user above 1000, ex. 1001 to inject the security context. Because this user does not technically exist, it also doesn't have a "HOME" directory.

To solve this issue, we were required to create an actual user with a home directory, and grant it permissions within the dockerfile.

This took much longer to discover than it should have, if the SDK would have errored out on a permission issue. Can you please take a look and consider throwing an error in this (and similar cases) within the SDK?

Thanks!

2

Hi D,

Thanks for posting. I am going to have someone dig into this because it does sound like it an on our end. I will try to get something in our next Sprint. Can you please open a ticket with Care and include a reference to this post and then post the Care ticket back here. I can then make sure doesn't get inadvertently closed and that you have visibility to our progress.

Thanks again.
John Carnell
Director, Developer Engagement

3

Hi John, here's the case number: 0003483780

4

Hi Dan,

I apologize I was traveling last week and wasn't as diligent in my forum posts. I did have our team have our ServOps team open a ticket and I have an engineer assigned to look at this on our next Sprint. Our next Sprint is targeted to start Tuesday, April 30th, but one of my engineers is winding down on work, so I have asked him to start digging into this now.

I don't think we have anyone try to run the SDK in a container context without a user associated with it so it might take us a little while to reproduce this in our lab. I will have our engineer @Hemanth post his questions here.

In the meantime, you have a couple of options:

  1. You can create a user directory as you did to work past this issue initially.

  2. You can call the APIs directly and bypass the SDK. It's not ideal, but our SDKs are nothing more than light wrappers around our calls.

  3. If you think you have identified the problem, you can always contribute the fix back. Our SDKs are open source.

If we can identify the problem we can usually get it turned around quickly.

Thanks,
John Carnell
Director, Developer Engagement

5

Hi @dandrivetime

Can you try using the latest version of SDK NuGet Gallery | PureCloudPlatform.Client.V2 201.2.0

We tried to address the exception and re-throw it instead of eating it.

Please let us know if you still see the excess CPIU usage issue.

Thanks
Hemanth

6

With NuGet Gallery | PureCloudPlatform.Client.V2 201.2.0 .I think it will throw error from catch block and how are you tackling this issue based on the error

7

Hi @Dileepkaranki

We're handling this by ensuring our non-root user in the container has a home directory, and we set all permissions accordingly. We raised the issue so the problem wouldn't be so hidden for others using the library, and to also ensure this was an oversight and we were on the right track with the solution.

Example, in our dockerfile (amongst other things our app needs):

RUN groupadd --system nonrootuser --gid 1001 && \ 
    useradd --uid 1001 --system --gid nonrootuser --home-dir /home/nonrootuser --create-home --shell /sbin/nologin --comment "non-root app user" nonrootuser && \
    chown -R nonrootuser:nonrootuser /home/nonrootuser

Then we do other stuff our image needs, copy files, etc...
Once we're near the entry point, we ensure permissions are granted to the directory we copied the app into (/app in this example):

RUN chown -R 1001:1001 /app

Hope this helps!

1 Like
8

Thank you, I appreciate your quick response

9

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.