• Response to https://twitter.com/simonw/status/1384383772255084546?s=20

    I still think it's rude to return MBs of data if your client device might be an inexpensive mobile phone, but for things like server-to-server API responses I May have been being way too early 2000s in my thinking Turns out returning 10+ MB of JSON works fine these days!
    I used to think similarly until I started thinking about compute in terms of carbon emissions (all data transfer/parsing/sending/storage requires electricity and thus likely carbon emissions for now). Since then I try to minimize all compute wherever possible.
  • Protocols Not Platforms

    The magic of web 2.0 were the open apis. Developers could use these apis to mashup services how they wanted. Sometimes these developer's tools and mashups became so popular that they would come to define the entire service of which they were building atop. Both hashtags and the term "tweet" originated outside of Twitter, Inc.

    But when these services grew they morphed into platforms. Their apis were closed off and the developers that helped these companies find their success either kicked off or severely limited in what they could do. This became a pattern, not just with Twitter, but many services that found success in thanks part to their open api followed the same playbook.

    Existing players making unpopular changes to their policies is usually a boon for the upstart. Each time this happens a vocal group of users becomes dissatisfied with the platform who then attempt to migrate to an alternative. However each migration causes some kind of loss. Data doesn't transfer or communities fracture because not everybody moves. Not to mention the energy that could have been spent doing something else.

    Contrast this with something like email. You can email anyone you'd like, even if they don't use the same provider as you. If your mail provider changes a policy you don't like, you're free to change providers without losing your identity on the internet. People can still contact you the way they always had and you can still contact them. Your data can move from platform-to-platform seamlessly. There's nothing re-organizing or hiding emails from your inbox unless you setup the rules (or use gmail).

    The difference in experience between twitter and email is night and day. One keeps you locked in and subject to their whims, while the other gives you the choice to use it however you see fit. The difference is that twitter is a platform and email is a protocol. Pick protocols.

  • Aggregating Nested Values with JSONFields

    Note: I originally wrote this for the Kwoosh Workshop blog. Since it's no longer around and I'm posting it here.

    Integrating with 3rd party APIs you'll often need to implement a webhook for your application to respond to external events. It's useful to take the event data and store it in your Database for future reference.

    As Kwoosh is run out of Texas, users in Texas need to pay an 8.25% sales tax. To my surprise Stripe's
    dashboard doesn't seem to offer an answer to a simple question: How much did I collect that was taxable
    and how much do I need to remit to the state for last quarter's sales.

    Armed with our event log data and Python we can quickly get that the numbers we're looking for.

    Our data looks something like this. Naturally a real event would have a lot more data, but for our purposes
    today this should suffice.

    "data": {
    "object": {
    "tax": 25

    This data is sent with each invoice.payment_successful event and it's saved in aa JSONField in our database. Using the KeyTransform we can extract values from our JSONField and annotate them
    to our QuerySet. Even better, as of Django 1.11 we can nest our KeyTransform so we can extract values
    that are multiple levels deep.

    Our plan of attack is to annotate the value, sum them together, and return them with the period.
    Unfortunately we have to sum them in Python until bug #29139
    is fixed. We're not summing millions of rows so it's still quick enough.

    from functools import reduce

    from django.contrib.postgres.fields.jsonb import KeyTransform

    from accounts.models import StripeEvent

    def accumulate_tax(start_date, end_date):
    Can't aggregate KeyTransformed objects directly, so summing manually
    Reference: https://code.djangoproject.com/ticket/29139
    events = StripeEvent.objects.filter(
    created__range=(start_date, end_date),
    tax=KeyTransform('tax', KeyTransform('object', KeyTransform('data', 'data'))),

    if events:
    tax_total = reduce(lambda x, y: dict((k, (v or 0) + (y[k] or 0),)
    for k, v in x.items()), events)
    tax_total = {'tax': 0}

    for key, value in tax_total.items():
    tax_total[key] = '${:,.2f}'.format(value / 100)

    return {'start': start_date, 'end': end_date, 'totals': tax_total}

    And just like that we can aggregate values from our JSONFields. If you want to sum other fields
    you can simply add another line like this, but replace "tax" with the field you want to sum.

    To make it a bit more useful, I built a second function that uses this function to give me quarterly sums.

    import datetime

    QUARTERS = (
    ((1, 1), (3, 31)),
    ((4, 1), (6, 30)),
    ((7, 1), (9, 30)),
    ((10, 1), (12, 31)),

    def calculate_quarters():
    today = datetime.date.today()
    last_year = today.year - 1

    for start, end in QUARTERS:
    quarter_start = datetime.date(year=today.year, month=start[0], day=start[1])
    year = today.year
    if quarter_start > today:
    year = last_year

    yield accumulate_tax(datetime.datetime(year=year, month=start[0],
    day=start[1], hour=0, minute=0,
    datetime.datetime(year=year, month=end[0],
    day=end[1], hour=23, minute=59,

    We can simply iterate over this function and generate a simple report that shows us a quarterly breakdown
    of our sales and taxes collected.

  • A short list of things cooler than Clubhouse

    Inspired by Seth's Chasing the cool kids, I've made a short list of things that are cooler than Clubhouse.

    • Not caring what the "cool" kids are doing and doing your own thing.

    • Building and learning about the systems that make the world go around.

    • Not uploading your entire contact list to some random company so you can eavesdrop on the "cool" kids.

    • Fighting for an open web so that you and future generations can access the world without gatekeepers.

    • Owning your content.

    The urgent advice usually ends with โ€œblogs are dead"

    If you always have to mention that "blogs are dead", perhaps they aren't actually dead. They never were dead. They're just not "cool" anymore. The people who made blogging cool and fun? They're mostly still blogging.

    Publish. Consistently. With patience. Own your assets. Donโ€™t let a middleman be your landlord. Yell at Google for blocking your emails and hope itโ€™ll work eventually. Continually push for RSS and an open web. With patience.


  • Bookmark of Cook: Companies like Facebook donโ€™t deserve praise, โ€œthey deserve reformโ€

    Technology does not need vast troves of personal data, stitched together across dozens of websites and apps, in order to succeed. Advertising existed and thrived for decades without it.

    From Tim Cookโ€™s remarks last week at Computers, Privacy & Data Protection 2021 conference. All I can say is: A-fucking-men. macOS may annoy me sometimes, but thereโ€™s no other company Iโ€™d trust with my location and health data.

  • Shipped my new webmentions dashboard in Tanzawa. You can see a preview of them on this post on the Tanzawa development blog.

    And we're live on prod!

  • With /r/wallstreetbets going private because the people are taking a bail out from the hedge funds, I can't help but think that we're on the first steps of a swing away from a centralized web. Finally. โœŠ๐Ÿš€

  • Just finished shipping a webmention endpoint in Tanzawa. Actual display of webmentions is a task for another time as it turns out to be quite complex.

  • Start a blog

    You should start a blog today by Juhis struck a cord with me and I thought I'd pile on. You should start a blog today.

    Like many I blogged a lot in the early 2000's. Those early blogs captured my frame of mind for that period, but they're long gone. Also like many, I stopped blogging sometime after Twitter and Facebook became popular.

    Over the years I tried to start back up again. "I should blog more", I'd tell myself. I'd always try to focus on writing "evergreen" content or writing "professional" content and after a short burst, stop.

    Discovering the IndieWeb helped remind me that I'm writing for me on my blog. It doesn't have to always be professional all the time. This past year or two regularly blogging again has helped me remember just how magical the internet is. That I can write something in Japan and people find it and respond to it from all over the world โ€“ all using open-standards โ€“ brings a smile to my face.

    Why should you start a blog today?

    • Develop better ideas. Many people develop their ideas by writing. They sit down with idea A and as the write about it, they gain some further insight and get idea B, which leads to idea C and so on. None of this would have been possible without sitting down to write. And you're not going to write unless you have a place to do so.

    • Be your own reference. When you're debugging a problem at work, chances are you're not the first person to run into that issue. Writing it down on your blog will not only help you gain a better understanding of the problem and help others solve the issue, but also in a year when you run into the same issue, you've got a refresher waiting for you on your blog.

    • Honest record of the past. Our memories aren't the best. Having a blog will help you remember just what you thought and felt, for better or for worse, when those events weren't so near.

    • Own your data. Twitter is a micro blog. Instagram is a photoblog. But these blogs aren't yours. Yes, you provide the photos. And yes, you provide the witty content. But all of it disappear in an instant at some company's discretion. Putting your data on your own blog protects you and your memories.

    It doesn't matter where you start your blog, or how cool your domain is, or how many people read it, or what programming language it's written in. What matters is that you start.

  • Really excited to have the first instance of ๐Ÿ”Tanzawa built and running on a server. Named after the mountain range I see everyday out my window.

1 of 5 Next