Uh oh!
There was an error while loading. Please reload this page.
What I've Learned So Far #10
Replies: 3 comments 3 replies
This is a fantastic write up! Thanks @benjamin-kirkbride for your very thorough/thoughtful observations. Just want to add 2 cents here and there (I may try for something more thorough at a later time):
He's referring to my talk at DjangoCon US 2022 titled Why I Didn't Start With Django. The short of it is: A lot of the things that are initially intimidating about Django end up being a moot point, considering all the things you have to learn when embracing other "micro" frameworks. Some annoyances (opinionated decisions) are unavoidable, but the ecosystem/community of Django is undeniably awesome.
Honestly, I actually went with a micro framework for the opposite reason most people do it. That is to say, I knew that if the "hello world" app could be built in 5 seconds, I would be spending a ton of time learning the "real world" skills to use the framework. But I actually wanted to know/learn how things were working from the ground up. I felt that Django, as powerful as it is, tends to obfuscate a lot of that with things like the ORM and/or class-based views.
I ended up using MongoDB for my blog. I happen to like MongoDB, particularly with its dictionary-like structure. Plus, it comes with the upside of no migrations (sort of). Considering that you are defining your data structures in advance, I don't get as frustrated with "messy" database design. It takes a little bit more discipline. MongoDB in Django is... not great.
I hardly have an opinion on this, but I wonder what it means for those who use other templating languages, such as Chameleon. (I haven't tried it myself, but know a handful of people who use it/like it.)
Interestingly enough, when I was re-factoring my blog (you know, for fun), I ended up building these classes... for my routes... that kind of look like... class-based ... views... Kind of.
I think it's fine. I spoke to people at DjangoCon that were drawn to Django for the same reasons I was initially repelled by it. It's great that Django exists for the large group of people that use it and adopt it wholeheartedly. In addition to things above, one thing that bugs me about Django is the As to using Django as default for PyHAT:
I still think it's worth exploring this. I may revisit this in another post, but hopefully we can get a sense of what the community wants around this subject! |
Could not agree more!
This is an excellent point, and I experienced this myself. My first web project, I used Django. I didn't really like it for some of the reasons I mentioned above, but a big reason I didn't like it was because I didn't feel like I really understood what was going on. My next project was FastAPI, and I loved it because I felt like I really understood what was going on. But then I started to not love it, because as time went on I realized I was spending a ton of time re-inventing the wheel remaking patterns that had already been solved by others in other frameworks (like Django) and fighting the ecosystem trying to make an MPA/HDA (though I didn't know that term at the time) in an ecosystem where that is an afterthought. It's a lot like learning how to use an ORM before you learn any SQL (another very common thing that people do). You can't (IMO) use an ORM effectively unless you have a decent grasp of SQL.
If you want to use another templating library with Django, you can, but you lose out on ORM/Form integration. This is a pretty high cost, especially if you are trying to make an HDA
My interpretation of @spookylukey's anti-class base view opinion (which I now share) is not that "using classes for views is bad" it's that the specific implimentation of Django's CBV's are bad, mostly because the obfuscate what is happening. A bad abstration. I don't think that class base views are inherently bad, and in fact I think if I was making a large Django app with a lot of views that were similar, I might make my own "class based view" using function based views 😅
Yes, this annoys me as well for sure 😅 but as you said, once it's done it's done and you really don't have to think about it anymore.
I couldn't agree more. Reading @florimondmanca's comment about how Symphony works in PHP-land, and it's relationship with other opinionated HTML centric frameworks, I think the Python ecosystem could really using something like that. A modern, modular and pluggable, but still opinionated and HDA-centric web framework would be awesome. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Congratulations on the write-up @benjamin-kirkbride, these are interesting observations. I had a couple of thoughts while reading this and going through django-htmx-patterns more thoroughly, to get a better grasp of the "What's the state of the template fragment technique", so I'll lay them down here. Behold, this is again going to be a mouthful of a comment! Hah. Reactions to post
I wouldn't say this is true for Symfony, although I have limited view of the community since I'm quite new to it. I would say this — for Symfony, Turbo is front of the scene because it was chosen as part of the symfony-ux initiative. This initiative is fairly new (2020) and aimed at providing a unified and well-integrated set of tools for dealing with JavaScript enhancements in Symfony. It also includes integration for Stimulus as well as a Symfony-specific Webpack integration (Webpack Encore). But I don't know how "mainstream" all this is today for Symfony folks. The tooling is fairly mature though, yes! Ruby (on Rails) seems to be the most ahead, but as you noted that is certainly linked to the commonality in people involved (DHH and team).
One extra reason (or synthesis of reasons) why I think Django might be better suited for HDAs than other microframeworks that allow displaying HTML, is exactly this — Django was made for displaying HTML. Django was made at a time when all web apps did really was displaying HTML, and then people added the small enhancements here and there using jQuery and the likes, until the industry realized we started doing so many enhancements why don't we let JS render HTML, and there begins the "lost decade". So, anyway, Django is really good at displaying HTML and providing all the tooling you need to build an HTML-centric web app. On the other hand, as you noted, microframeworks focused on JSON APIs, so their tooling revolves around that and not HTML. Django itself can't do JSON APIs very handily, except with the REST Framework or something.
It doesn't seem like CBVs are a must-use with Django today, are they? I'm sure you read it already, but there's a link in the django-htmx-patterns Just mentioning this as a point that might get the soundness of "I don't like CBVs" a tad lower still!
One thing that struck me when comparing Symfony with Django is the design philosophy is quite different. Both are opinionated in different ways. Django comes as a somewhat monolithic block, while Symfony has historically focused on modularity at the architectural level (not just "apps"). It's got Symfony Components which are "decoupled libraries for PHP applications". For example, the Form component can be used in isolation to Symfony as it basically deals with validating dicts of data (that typically come from forms) and binding those to a PHP POJO. I'd say, sticking to Symfony Components are what allows it to be the basis for other frameworks, such as Drupal or Laravel which both use its This puts the PHP ecosystem in a different position than Python. In Python, there's "Django with all things coupled into a tight and well-integrated pack" on one side, and "microframeworks perhaps with a modular framework as a shared base" on the other side (thinking of werkzeug + Flask for WSGI and Starlette + FastAPI for ASGI). But there isn't a framework akin to Symfony that provides components so widely reused and yet is a full-featured framework in its own right. The closest is Starlette, which was thought of as an "ASGI toolkit" and has highly reusable HTTP models and routing components effectively used as a basis by other frameworks (FastAPI is not the only one, many specialized microframeworks use Starlette as a toolkit). But Starlette stops there — it's an ASGI toolkit but not a full-featured framework with built-in components for forms etc (although Tom Christie initially aimed at developing a broader ecosystem, eg with typesystem or databases, at a time when async Python for the Web was just nascent). Template fragments -- State-of-the-art htmx pattern, vs Turbo
So I went through the {# monsters/templates/list.html #}{%extends"base.html"%}{%blockbody%}
<h1>List of monsters</h1>
{%ifpage_obj.paginator.count == 0 %}
<p>We have no monsters!</p>
{%else%}{%blockpage-and-paging-controls%}{%formonsterinpage_obj%}
<pclass="card">{{ monster.name }}</p>
{%endfor%}{%ifpage_obj.has_next%}
<pid="paging-area">
<ahref="#"hx-get="?page={{ page_obj.next_page_number }}"hx-vals='{"use_block": "page-and-paging-controls"}'hx-target="#paging-area"hx-swap="outerHTML"
>Load more</a>
</p>
{%else%}
<p>That's all of them!</p>
{%endif%}{%endblock%}{%endif%}{%endblock%}I initially wondered why there was so few "theory" explained Turbo. Its docs are very descriptive about "here's what Turbo can do for you", but they don't expose the same theoretical thinking than htmx does. I think it's because it embeds those principles without requiring the user to think about them much. So for example, with Turbo plus a bit of tooling, you'd leverage Turbo Streams, and write something like this: {# monsters/templates/list.html #}{%extends"base.html"%}{%blockbody%}
<h1>List of monsters</h1>
{%ifpage_obj.paginator.count == 0 %}
<p>We have no monsters!</p>
{%else%}{%blockpage-and-paging-controls%}{%formonsterinpage_obj%}
<pclass="card">{{ monster.name }}</p>
{%endfor%}{%ifpage_obj.has_next%}
<pid="paging-area">
<formmethod="GET"action="?page={{ page_obj.next_page_number }}"data-turbo-stream>
<inputtype="hidden"name="use_block"value="page-and-paging-controls" />
<inputtype="hidden"name="stream.action"value="replace" />
<inputtype="hidden"name="stream.target"value="paging-area /> <button type="submit">Load more</button> </form> </p> {% else %} <p>That's all of them!</p> {% endif %} {% endblock %} {% endif %}{% endblock %}(Click me) Why replace 'a' with a 'form method="GET"'?As you can see I'm using a button-triggered I had to decide where to put parameters we define in the template for LoB purposes: in the In general Turbo seems to get out of the way a bit more and encourage us to use the existing platform. Incidentally, htmx was born out of a desire to revisit the browser as a hypermedia platform, so it's interesting that despite the theory it doesn't simarly encourage leveraging existing HTTP APIs more. Its reluctancy to use Note: support for non-GET/POST form methods can still be achieved by sending an extra Then we could have a similar decorator helper, so the view ends up like this... @for_turbo_stream(use_from_params=True)deflist_monsters(request):
page_obj= ...
returnTemplateResponse(
request,
"monsters/list.html",
{"page_obj": page_obj},
)The
<turbo-streamaction="{{ action }}" target="{{ target }}"<template>
{{ block_content|safe }}
</template></turbo-stream>I'm sure there are many more use cases requiring slight adjustments to this pattern, such as explicit But that would be the tooling I'd need to reproduce what we're currently doing on a PHP/Symfony/Turbo project at work. It's interesting to note that if you look at the Turbo Streams docs, the approach shown there doens't have LoB properties. It uses a partial template with an Anyway, the point I wanted to make here was: my conclusion is that Turbo is more "higher-level" in that it doesn't just allow you to send requests with some other parameters on how to deal with the response (
Actually this was reckonized by the htmx folks themselves. See this Reddit comment 2y ago:
I don't quite agree with those claims, though. HTMX requires us to change the HTML quite dramatically, whereas as I've shown above, Turbo tends to be more transparent and help us leverage existing HTML APIs. For this exact reason, Turbo requires us to be precise and "expert" at the HTML we write, effectively getting down into the details as well. So, in conclusion, I think that while htmx provides crucial theoretical background (see LoB which informed me to go further than the Turbo docs in the example above), Turbo's "lack of flexibility" is actually a strong asset — it went the extra step to provide polished APIs, whereas htmx gives a whole bunch of attributes and it might be hard to figure out what the proper patterns are. And, as a meta-conclusion, I think this gives an extra reason that the "choice" of htmx over Turbo in the Django community is probably not so "informed", as it is "by chance". Turbo has been around for slightly more time and I don't think there's been as much "marketing" around it as htmx (the essays and memes etc helped a lot there). I do think Turbo has all the required pieces to also be a helpful tool for Django / Python folks when going the HDA route, perhaps helping us solve the "best patterns" problem by construction — since it's figured those out into more structured APIs for us already. |
@florimondmanca thanks so much for the detailed response. Sorry it took so long for me to respond, I have skimmed this a few times but never sat down to read it thoroughly.
No they are not, I should have clarified but that was a reason that I didn't want to use Django before I did all the research I ended up doing as part of this, at which point I read the same document you linked to and realized that was not a real issue as it's optional.
This is a really interesting information that I did not realize. It is really a shame that nothing like this exists for Python. I do appreciate the highly opinionated status of Django, but I don't like all the opinions, nor do many others. I think if it was more modular it would be more popular. Even the parts that are "modular" when you look deeper really are not, such as the templating engine. You can use Jinja, but you lose a bunch of the form/ORM integration.
I'm not 100% sure that this is the case? The example you gave could have been implemented using a form, though looking through https://htmx.org/examples/ I don't see an exact example that matches the scenario. I'll need to give this more thought and maybe talk to CG1 about it.
Tangent, but it's crazy to me that so few web tools support the full range of HTTP verbs to this day 😅
I agree with this for sure, Carlson Gross has enabled a paradigm shift in the way that a lot of people think about these things, regardless of if his specific tool for using it is the best solution. He has stated in many places that he thinks that HTMX is more or less just how HTML should work, but it's not necessarily the best library/pattern for achieving a HDA today.
This is probably the case, thinking about it more and in the context of @spookylukey's comment below.
I am going to have to reflect on this for a while. I really love HTMX but I will admit I haven't really given Turbo/Hotwire (or Unpoly) a fair shake. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
My (unmeasured) guess is that htmx was already more popular than Hotwire in the Django community when I started making that guide. Other people like Adam Johnson were also helping to popularise it. One factor in my choice is that Hotwire/Turbolinks comes from the Rails camp, and they tend to have a more "convention over configuration" / "magical-just-works" approach, while Python devs including myself tend to prefer "explicit over implicit". I was concerned that Hotwire would likely be a much better fit for Rails, while htmx was more framework agnostic. I remember seeing one tweet that backed that up, can't remember where now. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Just as a data point... — I looked into the Hotwire stack at some depth before settling with HTMX. I feel the choice was informed by more than just memes — which is not to say Carson doesn't have an A-grade Meme Game 😜 There was/is a turbo-django project https://github.com/hotwire-django/turbo-django which was OK, but to my eye didn't have the right flow to the API it wanted to offer. There were then, also, little inconsistencies with standard Django patterns, such as redirecting to a detail view after a form action, which made the turbo (and stimulus) integration not (at all) natural for Django development. On this last, the prospect of getting changes into the upstream project looked daunting — painful and slow at best, not possible at worst — and that was big warning flag for me. In contrast, integrating HTMX was a breeze. Less than an hour, and only a few attributes later (which speaks for the small changes to HTML point…) one is successfully doing cool things — entirely with the grain of existing Django workflows. I would say that is why it's been so readily adopted. Aside: I see good things with ASGI for the 'A' here. As I commented to @tataraba on Mastodon, I've thoughts around a similar 👍 |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Intro
I have spent the last few days at the PyCon sprints pouring over any materials I could find that even tangentially relate to hypermedia driven applications. This document is a mind-dump/attempt to get people up to speed on my findings and start some conversations.
The State of Hypermedia Driven Applications
It should come as no surprise that Pythonista's are not the only group that suffer from JS fatigue/aversion. In my readings, I came to two realizations:
I was partially turned on to this by @florimondmanca.
Both PHP and Ruby (on Rails) are ahead of Python when it comes to HDA use, and they both primarily use https://hotwired.dev/ (by the RoR people) to do it, primarily. Ruby on Rails and Symphony for instance both have very mature and mainstream use of the Hotwire tooling to turn them into true HDA frameworks. Their use of htmx is significantly less, compared to Hotwire and also to the use of htmx in the Python ecosystem.
To restate the above, HDA's in PHP and Ruby are now relatively mature and mainstream, almost entirely using Hotwire.
An aside, Hotwire does not use the terminology "HDA" to describe itself, but I find it to be a good term to use to compare to MPA and SPA.
Unpoly is another library in the same space that I am even less familiar with.
htmx
Coming back to htmx, it seems to be most prevalent in the Python community, and specifically within the Django community. This is a point I want to drive home; based on completely subjective gut feelings from all of the reading that I have done and evaluating of various frameworks, I would guess that >50% of htmx use is with a Python backend, and I would guess of that, over 70% is Django.
Factors that lead to me thinking that Python is the #1 ecosystem for htmx:
Factors that lead to me thinking that the overwhelming majority of htmx use within Python ecosystem is in conjunction with Django:
Django
The above conclusion about Django came as a surprise to me, as it's a framework that I had personally largely written off. This sentiment appeared to be shared by many that I knew and talked with at events like PyCon. In retrospect though, I shouldn't have been surprised that it would be a natural fit for htmx, and also that the Django community would be drawn to it.
Disclaimer
Everything I'm about to say is my opinion (or opinions that I used to have before starting on this journey), and should not be taken as either an endorsement, nor opposition, of anyone's choice to use any tool, framework, ecosystem, etc.
Django is not Cool
FastAPI is cool, le new Rust ASGI microframework is cool, even Flask/Quart is cooler than Django. You know what else is "cool" though? And also very common to use with the aforementioned micro-frameworks? JS heavy SPA's like React (more on that in a bit).
If you are a dev that stuck with Django throughout the lost decade, despite most of the mindshare of Python moving on to exciting new async micro-frameworks in that time, then you are probably the kind of grug brain dev that htmx was built for.
Django is not for SPA's
If you use Django for a project in the 2020's, you are almost certainly not using an SPA, and instead are making an MPA (DRF not withstanding). Likely you are doing this because you don't want to write much, if any, Javascript. If you are making an MPA, then you are likely making an HDA (or something very close to it), and even can gain quite a bit from literally just dropping
hx-booston your<body>tag. EZ PZDjango Has Everything You Need
Looking through https://github.com/spookylukey/django-htmx-patterns, the only thing that is recommended that doesn't come with Django stock is https://github.com/clokep/django-render-block, which wasn't even made for htmx/HDA's! The Django ecosystem had already built all the tools HDA's before HDA's were even a thing (:bangbang:). Django had this problem locked down years ago, and has been able to use htmx in a sane way the whole time.
It's also important to consider that this isn't just about the ability to maintain Locality of Behaviour in templates. Django, and the patterns and ecosystems around it, are built to ship hypermedia. Compare this to something like FastAPI, which is expressly built to make RPC style JSON API's (not to be confused with RESTful API's 😉) fast and easy to make. Templating tends to be a second class citizen in modern Python microframeworks.
If you aren't sure why someone might choose Django today to make a HDA over any given Python microframework, I would encourage you to read through https://github.com/spookylukey/django-htmx-patterns and consider what would need to happen to implement all of the patterns your favorite ASGI microframework (certainly not impossible! but not exactly trivial or in the spirit of the framework likely was intended for).
A lesson can be learned here from the non-Python Ecosystems with a vibrant community of HDA applications: Ruby on Rails, Symfony, Phoenix; do these look more like FastAPI or Django? Are people making HDA using Express or Sinatra? No. Because they are microframeworks made with serving JSON RPC API's to front end SPA frameworks and you have to fight the tooling and the ecosystem to use them for anything else.
Why Did Django Go With htmx Over Hotwire or Unpoly?
I am actually not sure. Ruby on Rails went with Hotwire because the Ruby on Rails people built it with RoR in mind, so that is no mystery. PHP uses it because Symfony built integration with it. Perhaps the reason Django didn't go with it is simply because the Django project is agnostic about it? It also could be because https://github.com/spookylukey/django-htmx-patterns was written by a Django core dev (@spookylukey) and that was enough to move the needle. (see
Why Don't People Want to Use Django?
Honestly not sure, broadly. @tataraba gave a talk about this. Speaking for myself though:
manage.pyfeels weirdurls.py)Reflecting on this list, none of these really hold much weight.
Takes longer to get started
If anything project that takes more set-up to get started than 30 seconds of pip install + copy example is written off as too complicated, then we are really in a sad state aren't we? I understand that low barrier to entry and such are important for adoption, but we are talking about decisions that will take up potentially decades of people's lives building and maintaining a given system. A decision like that should be made with more contemplation than dismissing options without thought because they take some time to give an honest evaluation. In fact, everything takes time to evaluate; the ability to go from 0 to
hello worldin 10 seconds flat is not any kind of signal of the long term viability of a platform.Soundness: 0/5
manage.pyfeels weirdmanage.py is used for two things, broadly (to my understanding):
Initializing a project has to happen no matter what, is using manage.py worse than cookiecutter or cloning or copy-pasting a bunch of stuff? Also, it starts every Django project on the exact same footing, with the same folder structure etc. Isn't that exactly what I wanted to try to facilitate by creating PyHAT; strong conventions so people can stop wasting/duplicating effort trying to solve the same problems slightly different ways?
Every project I have ends up with a
scriptsdir, may projects have a makefile for such things. Why am I upset that Django has a common tool for this? Also, pretty sure it's optional to use it, and even if it's not I can still have ascriptsdir that callsmanage.pySoundness: 0/5
I don't want to use a framework specific ORM
I really really don't, and I want to use a framework specific migration library even less. And it really doesn't seem optional, unless you are willing to throw out the baby with the bathwater when it comes to a lot of things, like
ModelForms.That said, almost everyone in Django uses it, so while it's technically bespoke, it's not exactly uncommon. Also, if you really need you can drop down to SQL. I think that this is less of a problem than it feels like to me, but more thought and experimentation is needed on my part.
Soundness: soft 3/5
I don't want to use a framework specific templating language
Literally everyone says "If you don't have a strong reason to use Jinja2, just use the standard built in one. I don't have a strong reason to use Jinja2, so...
Worth noting, Django does officially support using Jinja2. It's not clear to me exactly what you lose if you do that, again more experimentation is needed.
Soundness: 1/5 because it annoys me I even have to think about this
I don't like class based views
I'm not the only one, but seems like using function based views works just fine, though when reading docs or articles it seems like you may end up needing to understand both at least to some extent. Still, not great that such a big change was made that seems to have been such a bad call.
Soundness: 1/5
It's opinionated
Don't be a hypocrite. Chesterton's fence.
Soundness: 0/5
Do I Have Good Reasons for Not Using Django?
No, I do not
Does That Mean You Think Everyone Who Wants To Use htmx With Python Should Use Django?
No, but it does mean that I think two things:
There is nothing stopping a microframework from being great to use with htmx
It just so happens that basically every microframework made in the last decade or so was built to make "REST" API's to serve as the backend for an SPA, and that is basically all that the community around them is using them for, so it's an uphill battle.
Is it possible to make "the FastAPI for htmx" (taking suggestions for names now) that uses Jinja2, and SQLAlchemy, and has a 5 line example to hello world is 30 seconds? Sure, but honestly I think it would end up looking a lot like Django (or probably more like Symphony).
All reactions