Uploaded image for project: 'Apache YuniKorn'
  1. Apache YuniKorn
  2. YUNIKORN-2287

Decompress function doesn't need to decode base64

    XMLWordPrintableJSON

Details

    Description

      I'm working on https://issues.apache.org/jira/browse/YUNIKORN-2267. I added an example as following, but compression configmap doesn't work.

       

      1. Use gzip to compress and use base64 to encode the config

      ```

      echo "
      partitions:
        - name: default
          placementrules:
            - name: tag
              value: namespace
              create: true
          queues:
            - name: root
              submitacl: '*'
              queues:
                - name: parent
                  submitacl: '*'" | gzip | base64

      ```

      2. Set the result to `queues.yaml.gz` in binaryData field.

       

      Finally, we can see an error log in scheduler:

      ```

      2023-12-22T15:32:15.913Z    ERROR    shim.config    conf/schedulerconf.go:458    failed to decode schedulerConfig entry    {"error": "illegal base64 data at input byte 0"}
      github.com/apache/yunikorn-k8shim/pkg/conf.Decompress
          github.com/apache/yunikorn-k8shim/pkg/conf/schedulerconf.go:458
      github.com/apache/yunikorn-k8shim/pkg/conf.FlattenConfigMaps
          github.com/apache/yunikorn-k8shim/pkg/conf/schedulerconf.go:495
      github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).triggerReloadConfig
          github.com/apache/yunikorn-k8shim/pkg/cache/context.go:496
      github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateConfigMaps
          github.com/apache/yunikorn-k8shim/pkg/cache/context.go:394
      k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate
          k8s.io/client-go@v0.27.7/tools/cache/controller.go:250
      k8s.io/client-go/tools/cache.FilteringResourceEventHandler.OnUpdate
          k8s.io/client-go@v0.27.7/tools/cache/controller.go:315
      k8s.io/client-go/tools/cache.(*processorListener).run.func1
          k8s.io/client-go@v0.27.7/tools/cache/shared_informer.go:971
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
          k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:226
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil
          k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:227
      k8s.io/apimachinery/pkg/util/wait.JitterUntil
          k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:204
      k8s.io/apimachinery/pkg/util/wait.Until
          k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:161
      k8s.io/client-go/tools/cache.(*processorListener).run
          k8s.io/client-go@v0.27.7/tools/cache/shared_informer.go:967
      k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1
          k8s.io/apimachinery@v0.27.7/pkg/util/wait/wait.go:72

      ```

       

      I think the root cause is that the golang object is already decoded, so we don't need to decode it again.

      Attachments

        Issue Links

          Activity

            People

              yangpoan PoAn Yang
              yangpoan PoAn Yang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: