-
I Didn't Know iCloud Photo Was a Thing
byWhen sharing photos at work most of my co-workers would simply post a link to Google Photos in our company Slack. As an iCloud user, I thought my photos were only visible on my Mac or iPhone - machines logged in to my Apple account and setup to sync photos. So if I wanted to share photos with co-workers on Slack, I had to either upload them to Flickr or upload them directly into Slack. I always just uploaded them into Slack.
I just realized today that I can view all of my photos from the web via iCloud Photos. What’s more is I can share photos with a URL just like my co-workers have been with Google Photos. The shared link also expires after 1 month, which is a nice additional security / privacy feature.
Knowing that I can access my photos outside of Apple devices eases my mind. While I can’t ever see myself switching to Android from iOS, I could see myself using a Thinkpad + Linux for my desktop computing needs.
-
Using MarsEdit with DigitalOcean One-Click Wordpress Installs
byMy website has recently come full circle back to Wordpress. It’s been a number of years since I’ve used Wordpress. The last time was probably in college on the cheapest shared host I could find. I avoided coming back to Wordpress because I didn’t want to maintain a server; I fiddle enough with them at work. Already being a Digital Ocean customer, the 1-Click setup/hardened server seemed like the best way to go.
I quickly got it configured with all of the IndieWeb plugins to facilitate back-feeding content that I create on other platforms onto my website. The final step starting to use MarsEdit, my old favorite blog editor. Except it couldn’t connect to my website.
Turns out the reason is that the majority of Wordpress security issues stem from bots abusing the xmlrpc api and the digital ocean install blocks it at a low level by default. Disabling this block on the server allows programs to use the xmlrpc api and hence MarsEdit to work. Execute the following commands to disable the xmlrpc block.
sudo a2disconf block-xmlrpc
sudo systemctl reload apache2 -
by
Back when Macs came with modems built in, the one feature that I thought was super cool but never used was the Print to Remote Fax feature. It was one of those features like "Duh, of course. We can create PDFs. Naturally we can send them over the modem for printing."
-
An Overview of PDF Libraries in Python
byThe PDF file format is a ubiquitous file format and the only real way to guarantee document layout and display, no matter the device or operating system used to view it . Python, being a language that excels at data processing and manipulation, naturally has a lot of tools for working with PDF files to generate reports. This post is a brief introduction to the tools that are available and the tasks they work best for.
Wand
Wand is a set of Python bindings for do-it-all ImageMagick library. As such it really a Swiss-army knife for PDF (and image) manipulation and generation. In the context of PDF, ImageMagick (and therefore Wand) is mostly a frontend for GhostScript.Its API is quite Pythonic and can help you split, convert, and even draw on PDF files. Wand is perfect for manipulating existing PDFs and doing light editing on them.
ReportLab
ReportLab is a commercial/open-source PDF generation library. If you're making complex PDFs from scratch this is the library you should pick. Period.PikePDF
PikePDF is a PDF manipulation library based on qpdf. As its underpinnings are written in C++ it's super fast. Unlike other python pdf libraries, PikePDF supports unlocking encrypted / password protected PDFs.PikePDF also lets you explore the internal structure of a PDF document, which make it a perfect tool for debugging problem PDFs and extracting content/images.
The APIs are very Pythonic, and let you work with manipulate pages / page order like a regular list. PikePDF is also great for repairing and prepping PDF to be manipulated by Wand.
Pillow
Pillow is a PDF library, but it supports exporting images to PDF, making it a powerful tool in your Python PDF toolbox.While there's some overlap of functionality Wand in terms of image editing capabilities, it's more focused on editing images, rather than transforming and converting images. Pillow is an indispensable tool when working with any user-supplied images in your PDF workflow.
Conclusion
There is no single PDF tool in Python that fits all needs, all the time. These libraries can, and are, often used in conjunction with one another to quickly and easily generate PDFs on demand. However, depending on how you combine them, there can be some issues resulting in large file sizes and slow generation times. I'll cover these gotchas in future posts.
-
Unit Testing django-graphene GraphQL endpoints with pytest
byOn a project at work I've been learning GraphQL. I'm in charge of both developing the backend ( using the wonderful graphene-django package) and the frontend ( using Typescript / Vue.js / axios ) for this specific feature.
After wrapping my head around GraphQL and getting a basic endpoint working I wanted to write some tests to ensure that filtering and other requirements are working as expected.
For a proper End-to-end test I'd usually setup a client and post a json dictionary or such. With End-to-end GraphQL tests you need to send actual GraphQL queries, which makes sense but it feels a bit like SQL Injection.
Below is how I've settled organizing and writing my GraphQL tests.
tests/
├── __init__.py
├── conftest.py
├── test_myapp/
│ └── test_schema.pyBecause the graphql client will be reused across the django site, I added a global fixture that will automatically load mty project's schema.
# tests/conftest.py
@pytest.fixture
def graphql_client():
from myproject.schema import schema
from graphene.test import Client
return Client(schema)In this example I'm testing that I'm filtering data as expected when passing a search parameter.
For setup, first I write a query as its own fixture so I can re-use it throughout the test and it's clear exactly what is going to be run. Second, I make sure the query uses variables instead of hard-coded values when querying so I can change the input depending on the test. Third, setup a model_bakery fixture for data that I expect to find.
import pytest
from model_bakery import baker@pytest.mark.django_db
class TestMyModelData:
@pytest.fixture
def query(self):
return """
query testGetMyModel($searchParam: String!){
myModelData(searchParam: $searchParam) {
totalCount
}
}"""@pytest.fixture
def my_model(self):
baker.make(
"myapp.MyModel",
total_count="20", # Decimal field
)def test_none_response(self, graphql_client, query, my_model):
executed = graphql_client.execute(query, variables={"searchParam": "skittles"})
assert executed == {"data": {"myModelData": None}}def test_filters_usage(self, graphql_client, query, my_model):
params = {"searchParam": "skittles"}
executed = graphql_client.execute(query, variables=params)
assert executed == {
"data": {
"myModelData": {
"totalCount": 20
}
}
}Executing each test I simply pass my query and required variables for the test/query. In this I'm testing the same query twice: once with and one without a searchParameter. My expectation is that I get no results without a search term and data when to my graphql_client fixture.
As the return value from our client is a dictionary, I can simply assert my expecte results with the actual results. If something changes I'll know immediately.
Using the techniques above I can easily add new tests for my GraphQL endpoint as the available changes or bugs are found.
-
The Slow Web
byIt's been almost 5 years since I wrote Slow is not a Dirty Word. Reflecting on the sentiment in that article, that the best things in life take time and we needn't rush as society tries to force us, didn't quite go far enough. The concept of Slow should also be applied to the web.The Slow Web
What is the slow web? At it's core it's the idea that we shouldn't fill our mind with junk and we should connect with those around us. Social media is fast food for the mind. Consuming it feels in the moment, but when you look back you're not left with anything memorable. Moreover, because of the lack of nuance afforded by platforms, such as Twitter, it encourages behavior based on dopamine and adrenaline impulses.- The slow web is formulating your thoughts and expressing them fully.
- The slow web is about owning what you produce.
- The slow web is open.
- The slow web is yours.
The Slow Web in Practice
Most people the slow web is best manifested as a blog. This could be a simple Wordpress blog, a micro.blog, a bunch of static files on a server somewhere, anything that works for you. The important part is that you have control of your content. That you can control how and when it appears.Sharing
So much of sharing on the web these days is based on these social media platforms. So how do you get the word out about your new latest pieces in the slow web?- Writing unique content that matters to you and like-minded people will find it via search
- RSS Feeds (standard in most all blogging systems)
- POSSE (Publish (on your) Own Site, Syndicate Elsewhere
Ignore the Numbers
Knowing the number of visits to your website or article only serves to feed disappointment when one article doesn't match your expectations. Avoiding that sense of failure will inevitability lead to a habit of not writing and only consuming.Privacy
The common methods of tracking visits can not only break your site, it also invites an invasion of privacy for your readers. Are there more privacy-minded ways to collect visitor statistics? Yes. Do you even need to collect the information in the first place? Probably not.Don't Over Engineer
As technologies it's often easy to get caught up in nuts and bolts. We're want to build our websites to handle all the traffic the world can throw at it, so we setup database servers, build servers, deploy servers, proxy servers, and CDN caches. And for what? A trickle of traffic? All of this could be easily served off a single server, reducing operational complexity and reducing the places where things can break when you really just want to publish a blog.Making the Jump
Making the jump to the slow web doesn't mean you cannot participate in the social networks, you're just changing the terms of engagement. Instead of being the default place to collect your thoughts and ideas, it simply becomes another channel to link back to your site.Because you no longer tweet every clever thought you have into the void, you're able to slow down your mind, formulate your thoughts, and take back control. -
Recent End to End Testing Preferences with Django
byWhen testing in Django there's two basic ways to make an End-to-End test for your view: use the test client to send a request to the server or create a fake request object and manually call your view function.
One isn't "better" than the other, but I've come to prefer using the mock client over the fake request for the following reasons:
- Client tests hit the entire stack of code before executing your view allowing you to catch any conflicts with a middleware or settings and your view.
- Url Path tests come for free. When testing with fake request objects you can put any path you'd like in there and it will execute missing that bad merge where your url config change removing an endpoint.
- It's (slightly) easier to reason about. If I'm writing a test to confirm X happens when Y is posted I make Y and post it rather than making an object that pretends Y was posted.
- It removes the friction to refactor your views. As long as the url stays the same, you can rename and move your view however you'd like without changing any of the tests. This makes it easier to create a more consistent codebase e.g. some views use the verb "save" while others use "register".
-
Quality isn't an accident
byHow many times have you tried to plug in a USB cable and failed on the first try? If you’re like me that number is nearly the number of times I’ve ever attempted to connect a USB device. How many times has this happened with a network cable?While most people have heard of the term kaizen, or continuous improvement, when talking about Toyota and it’s lean manufacturing principles, fewer have heard of kaizen’s unsung sibling poka-yoke, or mistake proofing.
The main idea of poka-yoke is to draw attention to human errors as they occur, so they can be corrected or prevented, reducing the number of defects in a product. After observing a defect, we identify the root cause of the mistake, and then apply a fix or change in process that attempts to prevent the same error from happening again. This learning can then be applied to improve the design and function products.Building your products with poka-yoke requires more initial effort, however the end result is a higher quality and more reliable product. As developers, our tool to poka-yoke our services are unit tests.
Tests are small pieces of software that exercise a piece of code and verify that it’s behaving as intended. Writing tests are, however, many developers least favorite thing about creating software. We like to build new services and libraries because they’re fun and there is a sense of immediate gratification.
While writing good tests is time consuming and not fun. Setting up the scaffolding, defining test cases, and setting the test data can feel like tedious grunt work without an end in sight. However boring tests may be, they are essential to quality software.
With Kwoosh we’re aiming for high reliability, minimum regressions, and fast operations. Part of our testing strategy is to have “workflow” integration tests. These tests aim to help make certain that our tests are comprehensive while being easy to understand.
Each test currently has a maximum of two tests named ‘test_get_workflow’ and ‘test_post_workflow’. Each test then calls a series of descriptively named helper functions to assert behavior.def test_get_workflow(self): # Make sure that anonymous users are redirected to login # and that users are denied access to data that's not theirs self.redirect_for_anon_get() self.not_found_for_wrong_account() # Check that our filter interface is filtering data properly so # calls to "mine" only show tasks assigned to me and so forth. self.filters_response(self.sub_task_data['title'], 'all', []) self.filters_response(self.my_task_data['title'], 'mine', [self.sub_task_data['title'], self.completed_task_data['title']] ) self.filters_response(self.completed_task_data['title'], 'complete', [self.sub_task_data['title'], self.my_task_data['title']] ) def test_post_workflow(self): # Make sure that all post requests are redirect # or return 405 and denied self.redirect_for_anon_post() self.post_not_allowed()
As a lot of the tests will have some helpers in common (such as confirming anonymous users get redirected to login), we can bundle up these helper methods in small classes and include them into the tests, reducing the amount of code we need to maintain.
Less code = Fewer places for bugs = Less maintenance
Having descriptively named helper methods will allow future me to glance at the test I wrote today and identify exactly what’s being tested, without having to read through hundreds of lines of code.
Quality isn’t many things. Quality isn’t free. It isn’t flashy. It isn’t fancy. And it sure as hell isn’t an accident.