CRT Transfer Manager ignores endpoint URL scheme when determining SSL usage

[palakg11]

Describe the bug

When using the CRT-based S3 transfer manager with a custom HTTP endpoint, the CRT client incorrectly uses SSL even when the endpoint URL scheme is http://. This causes connection failures because use_ssl=True is hardcoded in _create_crt_client().

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The use_ssl parameter should be determined based on the endpoint URL scheme:

If endpoint URL starts with http:// → use_ssl=False
If endpoint URL starts with https:// → use_ssl=True
If no custom endpoint URL → use_ssl=True (default AWS endpoints)

Current Behavior

In boto3/crt.py, the _create_crt_client() function creates the CRT S3 client with use_ssl=True hardcoded, regardless of the endpoint URL scheme:

def _create_crt_client(session, config, region_name, cred_provider):
    """Create a CRT S3 Client for file transfer."""
    use_ssl = True  # Always True, ignores endpoint URL
    
    create_crt_client_kwargs = {
        'region': region_name,
        'use_ssl': use_ssl,
        'crt_credentials_provider': cred_provider,
    }
    return create_s3_crt_client(**create_crt_client_kwargs)

### Reproduction Steps

Try to do crt upload with http endpoint

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### SDK version used

1.42.16

### Environment details (OS name and version, etc.)

Ubuntu 22.04

[adev-code]

Hello @palakg11, thanks for reaching out. I have tried to upload using CRT with HTTP endpoint and I did not get the same issue. The HTTP protocol did not change to HTTPS.

For example:

import boto3
from botocore.session import Session
from s3transfer.crt import BotocoreCRTRequestSerializer, CRTTransferManager, create_s3_crt_client, BotocoreCRTCredentialsWrapper

boto3.set_stream_logger('')

session = Session()
serializer = BotocoreCRTRequestSerializer(session, client_kwargs={'region_name': 'us-east-1', 'endpoint_url': 'http://example_domain.com'})
creds = session.get_credentials()
crt_creds = BotocoreCRTCredentialsWrapper(creds).to_crt_credentials_provider()
crt_client = create_s3_crt_client('us-east-1', crt_creds)
manager = CRTTransferManager(crt_client, serializer)

with manager:
    future = manager.upload('README.rst', 'test-bucket', 'test.txt')
    future.result()

Could you provide a minimal reproduction code for the issue you are seeing? Thanks.

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

[palakg11]

@adev-code I added a monkey patch to change the use_ssl flag based on endpoint scheme. Import this file in your codebase for patch to apply

"""Monkey patches for boto3.crt to change default use_ssl behaviour.
sys.modules guarantees this code runs only once.
"""

import logging

_logger = logging.getLogger(__name__)

try:
    import boto3.crt
    from boto3.crt import create_s3_crt_client

    def _patched_create_crt_client(session, config, region_name, cred_provider):
        """Create a CRT S3 Client for file transfer.

        This patched version respects the endpoint URL scheme:
        - http:// → use_ssl=False
        - https:// → use_ssl=True
        """
        # Determine endpoint URL using boto3's resolution logic
        from s3transfer.utils import create_nested_client
        temp_client = create_nested_client(
            session,
            service_name='s3',
            region_name=region_name
        )
        endpoint_url = temp_client.meta.endpoint_url
        use_ssl = endpoint_url.startswith('https://') if endpoint_url else True # Default to True (HTTPS)

        _logger.debug("Creating CRT client with endpoint_url=%s, use_ssl=%s (patched)",
                      endpoint_url, use_ssl)

        create_crt_client_kwargs = {
            'region': region_name,
            'use_ssl': use_ssl,
            'crt_credentials_provider': cred_provider,
        }
        return create_s3_crt_client(**create_crt_client_kwargs)

    # Apply the patch
    boto3.crt._create_crt_client = _patched_create_crt_client
    _logger.info("boto3.crt patches applied successfully")

except ImportError as e:
    _logger.debug("Could not import required modules for CRT patches: %s", e)
except Exception as e:
    _logger.warning("Failed to apply boto3.crt patches: %s", e)