As part of my Markdown Virtual Table project, when running the development runserver , I occasionally want to run some code when we update a file.

There are two signals that we can use for this, that are not well documented in Django itself:

from django.utils.autoreload import autoreload_started, file_changed

These are mostly used internally with the development runserver.

autoreload_started

The first half of this is used to register our file watches. We can find some of these examples in Django itself.

I have been continuing with my virtual table implementation, though taking a short break to do some more integration on the Django side.

I have an ideas site that I use as a kind of wiki to keep track of various ideas.

# the default model I'm using from my virtual table integration
class FrontmatterModel(models.Model):
    inode = models.PositiveIntegerField(primary_key=True, editable=False)
    path = models.FilePathField(unique=True, editable=False)
    slug = models.SlugField()

    content = models.TextField()
    excerpt = models.TextField(blank=True)

    metadata = models.JSONField()

    class Meta:
        abstract = True

# Custom model for my idea object itself
# These also exist in the `metadata` json frontmatter, but putting them here
# makes them nicer to deal with from Django
class Idea(FrontmatterModel):
    title = models.CharField(max_length=128)

    status = models.ForeignKey(Status, on_delete=models.SET_NULL, null=True, db_column="dir")

    homepage = models.URLField(blank=True)
    repository = models.URLField(blank=True)
    focus = models.BooleanField(default=False)
    date = models.DateTimeField(default=timezone.now)

    class Meta:
        required_db_vendor = "markdowndb"
        managed = False

The model ultimately gets sent to the virtual table backend with a statement that looks like this

I have been continuing my earlier virtual table prototype and it is slowly coming along. I uploaded a version to codeberg while I was debugging python packaging. I do not have it quite working right, but I think I will be using setuptools-rust for now (pr ).

This post is partly to help document what I have discovered over time so will not be complete code examples in every case. In the future I will be uploading the project to Codeberg and later to PyPi.

Most of my rust code is likely not great, so would appreciate a mentor!

I wanted to have some drag and drop web components that I could use with my django apps. I am planning on using them in another of my many todo/scheduling prototypes. Basing heavily on the api docs for drag and drop, I start with some basic components.

I start with my <drag-source> component that I will use to wrap anything I want draggable.

class DragSource extends HTMLElement {
  constructor() {
    super();
    // I use a random ID here to make something quick/simple but in a real version
    // I should probably only set this if not already set.
    this.id = Math.random();
  }

  dragStart(e) {
    e.dataTransfer.setData("text/plain", e.target.id);
    e.dataTransfer.dropEffect = "move";
    setTimeout(() => {
      e.target.classList.add("hide");
    }, 0);
  }

  dragEnd(e) {
    e.target.classList.remove("hide");
  }

  connectedCallback() {
    this.addEventListener("dragstart", this.dragStart);
    this.addEventListener("dragend", this.dragEnd);
  }
}
customElements.define("drag-source", DragSource);

Next, I create a container that I will use to drop onto.

Sqlite has support for virtual tables which can be used for external datasources. There is an example csv data source that allows you to query a csv file as if it was a regular table.

.load ./csv
CREATE VIRTUAL TABLE temp.csv USING csv(filename=FILENAME);
SELECT * FROM csv;

I do not really want to write C for this, but I found the rusqlite project which allows writing a loadable_extension that provides virtual tables. Using their vtablog example (slightly modified) I was able to test from sqlite with a rust module.

Even though I wrote about it in a previous week note I have not been using my start page to much yet. Yesterday I randomly decided to start using it after cleaning a few things up. Combined with my dev server I had some of the pieces ready.

First, I configured Firefox to change my start page to start.localhost, where I have my page running. Unfortunately, while you can configure the home page, you can not currently configure the new tab page.

While updating my PowerPlug ( pypi ) project, I implemented a pattern I have started using in a few other places that I use entry-points .

By creating a custom enum using one of Django’s enumeration-types , you can make it much easier to load these modules.

from importlib import metadata

from django.db import models
from django.utils.translation import gettext_lazy as _

class EntryPoints(models.TextChoices):
    APPS = "powerplug.apps", _("Installed Apps")
    REST = "powerplug.rest", _("Installed APIs")
    SIGNAL = "powerplug.signal", _("Installed Signals")
    TASK = "powerplug.task", _("Installed Tasks")
    URLS = "powerplug.urls", _("Installed URLs")
    CONTEXT = "powerplug.context", _("Installed context processor")

    def group(self) -> metadata.EntryPoints:
        return metadata.entry_points(group=self.value)

Using an enum like this, makes it very easy to load one of our entry points, while keeping all the definitions for our app in a single place.

While experimenting with yamdl to prototype an idea for a wiki-like django site, I was curious about the various lower level Django signals. To better understand them, I wrote a simple app config to log the various checks. To help them be more visible in the terminal, I have also somewhat randomly added some assorted colors.

import os
from pathlib import Path

from django.apps import AppConfig
from django.core.checks import register
from django.db.backends.signals import connection_created
from django.db.models.signals import class_prepared
from django.dispatch import receiver
from django.utils.autoreload import BaseReloader, autoreload_started, file_changed
from django.utils.termcolors import make_style

red = make_style(fg="red")
blue = make_style(fg="blue")
cyan = make_style(fg="cyan")
yellow = make_style(fg="yellow")
magenta = make_style(fg="magenta")
bold_yellow = make_style(opts=("bold",), fg="yellow")


class TestappConfig(AppConfig):
    name = __package__

    def ready(self):
        print(bold_yellow("ready"), __class__, os.getpid())
        if os.environ.get("RUN_MAIN"):
            print(bold_yellow("On app thread"))


@register
def test_checks(*args, **kwargs):
    print(red("test_checks"), args, kwargs)
    return []


@receiver(connection_created, dispatch_uid="print_connection_created")
def print_connection_created(*args, **kwargs):
    print(red("connection_created"), args, kwargs)


@receiver(autoreload_started, dispatch_uid="print_autoreload_started")
def print_autoreload_started(sender: BaseReloader, **kwargs):
    print(magenta("autoreload_started"), sender, kwargs)


@receiver(file_changed, dispatch_uid="print_file_changed")
def print_file_changed(sender: BaseReloader, file_path: Path, **kwargs):
    print(magenta("file_changed"), sender, file_path, kwargs)


@receiver(class_prepared, dispatch_uid="print_class_prepared")
def print_class_prepared(sender, **kwargs):
    print(cyan("print_class_prepared"), sender, kwargs)

After adding my new app to my INSTALLED_APPS setting, I get a similar output when I run it.

I have several Django projects, like my timebox or chore tracker project.

Two fields I will often add to most models is some kind of description/memo field and a url/more field.

from django.db import models

class Project(models.Model):
    # most of the model specific bits
    ...
    # common fields I add.
    description = models.TextField(blank=True)
    url = models.URLField(blank=True)

I will typically allow either to be blank, since both are intended to be optional.

A name/title might be enough as a quick hint or label, but often having a longer description/memo field is essential to provide extra detail once you have enough objects. I have mostly kept it as a plain text description (I will often use {{ obj.description|linebreaks }} to allow a bit of formatting), though other times I will consider allowing a full markdown description.

A friend recently wrote “our personal database , about how they maintain a postgres database for various projects. I wrote something similar when I wrote about my personal api , though Rubenerd’s post made me want to write a little bit more about it.

Since I am already using django for various projects, my personal API is also built on Django. While I could modify thing directly using something like pgAdmin, building on Django means I can take advantage of the models and admin built in to Django.

There have been a few times while working with Django , where I have wanted to triger something only if the instance changed. Since Django provides a robust signals system, I decided to use that for my prototype.

First we need to import the packages we will use.

from django.db import models
from django.db.models.signals import post_save, pre_save, ModelSignal
from django.dispatch import receiver
from django.forms.models import model_to_dict

Next, we want to handle our pre_save signal. We want to store a snapshot of the original object so that we can compare it later.

One advantage of having my own personal api is that I can put various useful scripts under a single repo and have them run. I have been using raindrop for several years, to collect bookmarks to read later. Often, while researching things, it would be useful to automatically group things into collections, so I wrote some celery tasks to help with this.

I have long been inspired by Aaron Pareki and his pk3 tool for his website. With some searching, one can find other kinds of personal management systems on GitHub or other developers writing about their own personal api with links to other examples. As a developer myself, I have my own personal API that I am able to add to as wanted.

In the interest of choosing boring technology my personal api is powered primarily by django and celery .

Django comes with support for MIDDLEWARE and provides several useful ones by default. I usually try to make my projects as useable as possible, and some debug middleware is only useful when development.

Example Middleware

Since the order and layering often matter, I’ll usually configure all my optional middleware in the correct spot like bellow, with a short comment.

MIDDLEWARE = [
    "debug_toolbar.middleware.DebugToolbarMiddleware",  # Only enabled for debug
    "django.middleware.security.SecurityMiddleware",
    "whitenoise.middleware.WhiteNoiseMiddleware",  # Used primarily for docker
    "django.middleware.locale.LocaleMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
]

Then I’ll use other conditional checks to see if this list (or other variables) need to be modified.

I’ve been using hugo for my blog for a while, and while I appreciate being able to write posts in anything that supports Markdown, I’ve never enjoyed making sense of their template system. My site curently uses a fork of hugo-bootstrap (to fix minor bugs) with some other customizations from hugo-worklog that I wrote.

Ideally I would like a system where I can use django for the site, but maintain the same editing flow I have for my current site.

Our goal when packaging up a Django application, is that we can use it as part of an existing application, or we can run it by itself in a standalone mode. To this end, I have over time, started to package my django applications in the following way. You can see diffs of all the commits in the example-django repository.

Start with a Makefile and setup files

Instead of using tools like Poetry and Pipenv , I find it easier to just create a basic Makefile to use as the entrypoint.