While reading the release notes for a recent update of python-social-auth I was thinking about how I handle User-Agent headers in django apps.

requests is a popular library for Python that many projects use. By default, requests defaults to the user agent of python-requests/{__version__}

https://github.com/psf/requests/blob/v2.32.5/src/requests/utils.py#L881-L901

The above mentioned python-social-auth is built on requests, but it overrides it with it’s own value.

This means for most backends, when you login with social auth, it will send social-auth-{social_core.__version__} as it’s user agent.

My Implementation

For my own django-zakka project, I did something similar but a little different.

https://codeberg.org/kfdm/django-zakka/src/tag/v0.7.0/zakka/http/client.py#L13-L27

Using the version method from the importlib.metadata package, we can easily get the version of our library and use that to get our version number.

def user_agent():
    v = version(distribution_name="django-zakka")
    return f"{name}/{v}"

If I am running within the context of a Django project, it isn’t so useful to know that the client is using requests or social-core or django-zakka to make a request. It could be far more useful to know that the request is coming from foo-app or bar-app (without having to look at client ip_addresses in the logs).

DEFAULT_DISTRIBUTION = getattr(settings, "USER_AGENT_DISTRIBUTION", "django-zakka")

@lru_cache
def user_agent(name):
    v = version(distribution_name=name)
    return f"{name}/{v}"

USER_AGENT = getattr(settings, "USER_AGENT", user_agent(DEFAULT_DISTRIBUTION))

I also provide a wrapper around requests.Session that handles setting our user agent.

class DjangoSession(requests.Session):
    def __init__(self, distribution_name=DEFAULT_DISTRIBUTION):
        super().__init__()
        self.headers["user-agent"] = user_agent(distribution_name)

This allows me to set a value in Django’s settings.py so that my requests report the correct app in the user-agent.

Similar to how Mastodon requests report their server, my real version also optionally reports the domain.

from django.apps import apps
from django.contrib.sites.shortcuts import get_current_site
...
@lru_cache
def user_agent(name):
    try:
        v = version(distribution_name=name)
    except PackageNotFoundError:
        v = "unknown"

    if apps.is_installed("django.contrib.sites"):
        domain = get_current_site(None).domain
        return f"{name}/{v} (+{domain})"
    else:
        return f"{name}/{v}"

While working on my personal api I tend to have clients for a number of services that I use.

class ForgejoSession(DjangoSession):
    ...

class HomeAssistantSession(DjangoSession):
    ...

class RaindropClient(DjangoSession):
    ...

and so on.

Problems

This works great, as long as everything I do uses my library.

If I use social-core or requests or something else, it will go back to using the default library from their respective library.

It would be nice if there was a method provided by the requests library, that we could set our own default user agent.

Alternatively, it’s possible there could be a very small, lite wrapper around request that implements a requests.Session wrapper that also provides a set_default_agent() method. Libraries could then use this lite wrapper instead of requests directly.

The most realistic fix for now, may be monkey patching the few user agent methods in the libraries that one cares about. One would likely end up with some number of try/catch statements to check for various libraries to be monkey patched.

from importlib.util import find_spec

if find_spec('requests'):
    patch_requests()

if find_spec('social_core'):
    patch_social_core()

...

While many of these libraries are fairly stable, but monkey patching will always carry a risk of an upstream change breaking.

I do not have any specific call to action, but found it useful to write this up and look at how a few different libraries handled things.