Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-5482

Memory leak during SSL handshake

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.12.0
    • 0.16.0
    • C++ - Library
    • None
    • Build env: Windows 2019

      Issue observed: Windows 2012R2

      OpenSSL: 1.1.1k

    Description

      We observed continuous memory increase in the thrift server during SSL handshake and this memory increase was crashing the windows hosts as there was no memory left for other applications to work. During further investigation, we found that ERR_remove_state has been made stub in the later versions of OpenSSL, and recommended function to clear the thread state is OPENSSL_thread_stop. Once I made this change we did not observe any memory leak further. We are using OpenSSL 1.1.1k with thrift 0.12.0. Hence I am putting the patch for the same. Thanks!

      I am adding call stack sample leak trace

      
      Call stack sample 2
      Address   0x0000000e`00bdd770
      Allocation Time  00:00:45 since tracking started
      Allocation Size 64 Bytes
      
      
      Function
      LeakTrack+1f109
      !_malloc_base+44
      !CRYPTO_zalloc+59
      !BN_CTX_new+20
      !rsa_ossl_public_decrypt+d8
      !pkey_rsa_verify+245
      !EVP_DigestVerify+15f
      !tls_process_cert_verify+42a
      !read_state_machine+24f
      !state_machine+2d2
      !SSL_do_handshake+bc
      !apache::thrift::transport::TSSLSocket::initializeHandshake+dc
      !apache::thrift::transport::TSSLSocket::read+3c
      !apache::thrift::transport::TBufferedTransport::readSlow+56
      !apache::thrift::transport::TBufferBase::readAll+88
      !apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport,apache::thrift::protocol::TNetworkBigEndian>,apache::thrift::protocol::TProtocolDefaults>::readMessageBegin_virt+4c
      !apache::thrift::TMultiplexedProcessor::process+87
      !apache::thrift::server::TConnectedClient::run+16a
      !apache::thrift::server::TThreadedServer::TConnectedClientRunner::run+17
      !apache::thrift::concurrency::StdThread::threadMain+6c
      !std::thread::_Invoke<std::tuple<int (__cdecl*)(std::shared_ptr<Reducted>),std::shared_ptr<Reducted> >,0,1>+3e
      !thread_start<unsigned int (__cdecl*)(void * __ptr64)>+50
      kernel32!BaseThreadInitThunk+22
      ntdll!RtlUserThreadStart+34
      
      

      Attachments

        Issue Links

          Activity

            People

              anshulmgupta Anshul Mohan Gupta
              anshulmgupta Anshul Mohan Gupta
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m