https://wakatime.com/2024-01-01T00:00:00ZThe WakaTime Bloghttps://wakatime.com/blog/65-wakatime-2023-programming-statsWakaTime 2023 Programming Stats2024-01-01T00:00:00Z2024-01-01T00:00:00Z2 min read<p>Happy New Year!</p>
<p><img src="https://wakatime.com/static/img/blog/classical-wohnung-wien.jpg" class="img-thumbnail" alt="classical house vienna jugend herrenvilla" title="classical vienna jugend herrenvilla located at Peter-Jordan-Straße 23, 1190 Wien" style="width:90%" /></p>
<div style="font-size:10px;text-align:right;width:90%;margin-top:-18px;margin-bottom:10px;">
Photo by <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/@alan">Alan Hamlett</a>
</div>
<p>In 2023, more than 500k developers spent a combined 51 million hours programming, tracked with <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/plugins">WakaTime plugins</a>.
The average person spent 48 minutes per day programming.</p>
<p>Each year we release a <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/a-look-back-at-2023">personalized coding report</a> showing your accomplishments over the last year.
We also include average trends for all WakaTime devs showing the most used languages, IDEs, and operating systems.
These trends come from aggregating the WakaTime personal dashboard metrics when a developer installs one of our <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/plugins">IDE plugins</a>.
But first, a quick list of new features released in 2023.</p>
<h2>New Features & Milestones</h2>
<h3><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/61-create-invoices-from-your-wakatime-code-stats">🧾 Invoicing</a></h3>
<p>Create PDF invoices from your code stats.</p>
<h3><a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/macos-wakatime#readme">💻 Official WakaTime Mac App</a></h3>
<p>We’ve released our first desktop app, for Mac.
With the mac app, you get code stats for your desktop apps such as Figma, Postman, and Xcode.</p>
<h3><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/polls">🤨 GitHub Issue Polls</a></h3>
<p>Collect votes from your GitHub issues.
This mini-feature helps project maintainers collect feedback within GitHub issues.</p>
<h2>Programming stats for 2023</h2>
<p><a name="top-languages"></a></p>
<h4>Top languages for 2023</h4>
<p><img src="https://wakatime.com/share/all-users-stats/2023-languages.png" class="img-thumbnail" alt="top languages" style="width:90%" /></p>
<p><a name="top-editors"></a></p>
<h4>Top editors for 2023</h4>
<p><img src="https://wakatime.com/share/all-users-stats/2023-editors.png" class="img-thumbnail" alt="top editors" style="width:90%" /></p>
<p><a name="top-os"></a></p>
<h4>Top operating systems used</h4>
<p><img src="https://wakatime.com/share/all-users-stats/2023-operating_systems.png" class="img-thumbnail" alt="top operating systems" style="width:90%" /></p>
<p>See your personalized report for 2023 here:</p>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/a-look-back-at-2023" style="
font-size:16px;
font-weight:700;
padding:15px 40px;
color:#fff;
background-color:#2595ff;
border-color:#0b89ff;
text-decoration:none;
display:inline-block;
margin-bottom: 0;
text-align: center;
vertical-align: middle;
touch-action: manipulation;
cursor: pointer;
background-image: none;
border: 1px solid transparent;
white-space: nowrap;
line-height: 1.428571429;
border-radius: 4px;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;">View your personalized 2023 stats</a></p>
<p>These aggregated 2023 stats are <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/api/v1/stats/2023">available as JSON</a>, and also available for <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/tag/yearendreport">previous years</a>.</p>
Alan Hamletthttps://wakatime.com/blog/64-github-permissions-updateGitHub Permissions Update2023-09-22T00:00:00Z2023-09-22T00:00:00Z2 min read<p>We’ve improved the security of our <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/integrations">GitHub integration</a> by migrating to GitHub Apps, for fine-grained permissions and read-only repo access.</p>
<h2>Why GitHub Apps?</h2>
<p>Ever since the GitHub released <a rel="nofollow noopener" target="_blank" href="https://developer.github.com/changes/2017-05-22-github-apps-production-ship/">GitHub Apps</a>, <a rel="nofollow noopener" target="_blank" href="https://github.blog/2018-10-11-sentry-guest-post/">many</a> <a rel="nofollow noopener" target="_blank" href="https://github.blog/2018-09-25-migrating-to-github-apps-code-climate-shares-their-story/">companies</a> have transitioned from OAuth to GitHub Apps.
<code>GitHub Apps</code> is the term for GitHub’s new OAuth client implementation... it’s GitHub’s rewrite of their API’s OAuth authentication flow, but it’s still just OAuth behind the scenes.
The main reason to use GitHub Apps is fine-grained permissions.</p>
<p>When using GitHub’s API without GitHub Apps, the permission scopes are way too broad.
For example, want read-only access to source code?
That’s not possible without using GitHub Apps.</p>
<p>GitHub could have just saved some energy and fixed their existing OAuth scopes, but their product team didn’t ask me before deciding to rewrite the whole thing ;)</p>
<h2>What’s changed?</h2>
<p>Now that we’re using GitHub Apps, when you connect the <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/integrations">WakaTime GitHub integration</a> you will only be asked to grant read-only permission to your repos.
We don’t actually read your source code, just your commit messages.
But, having access to commit messages means GitHub also grants read-only access to repo contents.</p>
<h2>Onboarding changes</h2>
<p>With the old OAuth app, users would grant access to the integration in one step, called the OAuth consent screen.
However, with the new GitHub Apps after the OAuth consent screen redirects back to wakatime.com, users are redirected to a second GitHub Apps install consent screen.
This GitHub Apps install page doesn’t use the OAuth <code>redirect_uri</code>, it always redirects back to the <code>Post installation setup url</code> defined in the GitHub Apps settings.
It also only redirects back if the user makes changes to the GitHub App’s permissions or selected repository access.
That means the onboarding flow breaks if we redirect to the GitHub Apps install page when the app is already installed and no changes are necessary.
For that reason, after you first authorize the WakaTime GitHub App you’re only redirected to the second GitHub App consent screen if you don’t already have the GitHub App installed.
To find that out, we <a rel="nofollow noopener" target="_blank" href="https://docs.github.com/en/rest/apps/installations?apiVersion=2022-11-28#list-app-installations-accessible-to-the-user-access-token">list your WakaTime app installations</a> using the GitHub API and only redirect to the second consent screen if we don’t find any installs.</p>
<h2>Changing the repos WakaTime can access</h2>
<p>If you need to edit the repo permissions you’ve granted WakaTime access, click the <code>Adjust GitHub permissions</code> link on the <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/integrations/github">GitHub integration settings</a> page.</p>
<p>This migration to GitHub Apps has been something we’ve wanted to do for a while now, and I’m pleased we’ve improved the security of the WakaTime GitHub integration for our users.
Happy coding!</p>
Alan Hamletthttps://wakatime.com/blog/63-tracking-compile-times-with-eclipseTracking Compile Times with Eclipse2023-08-30T00:00:00Z2023-08-30T00:00:00Z1 min read<p>The <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/eclipse">Eclipse plugin</a> now tracks how long your builds take to compile.
Shortest WakaTime blog post ever.</p>
<p>Happy codin…, I mean happy compiling!</p>
Alan Hamletthttps://wakatime.com/blog/62-xcode-supported-againXcode Supported Again2023-03-09T00:00:00Z2023-03-09T00:00:00Z1 min read<p>Ever since Apple <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/25-xcode-8-will-disable-all-your-plugins">disabled plugins</a> in Xcode, we’ve been looking for a way to still show your code stats for Xcode without a plugin.
Now, we’re happy to announce a native Mac app that adds back support for Xcode code stats!</p>
<p><a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/macos-wakatime">WakaTime for Mac</a> runs in your system tray and uses Apple's accessibility features to watch for typing in Xcode.
The time you spend coding in Xcode is now displayed on your <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/">WakaTime Dashboard</a> again.</p>
<p>Today we <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/macos-wakatime/releases">released</a> the first working beta version.</p>
<p>Give it a try, and let us know what you think!</p>
Alan Hamletthttps://wakatime.com/blog/61-create-invoices-from-your-wakatime-code-statsCreate Invoices from your WakaTime code stats2023-02-26T00:00:00Z2023-02-26T00:00:00Z1 min read<p>As a freelancer, I’ve spent years juggling multiple projects, filling out invoices, and trying to keep track of how much time I spend on each task.
This often resulted in me forgetting some billable hours, causing frustration and affecting my income.
Today I’m pleased to announce a solution: WakaTime Invoicing.</p>
<div style="position:relative;margin-top:30px;margin-bottom:30px;padding-bottom:57.23370429252782%;height:0;"><iframe src="https://www.loom.com/embed/f4203c3370214fcabd9b217a9cb1ecb5" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/invoicing">WakaTime Invoicing</a> utilizes your WakaTime code stats to detect the projects you’ve worked on, and the amount of time you’ve spent on each project.
It then generates your invoice automatically, with the hours you worked on each project.
This means you can simply focus on your work and let WakaTime keep track of how long you worked.
When it’s time to bill your clients, WakaTime generates your pdf invoice and you can be confident that you won’t miss any billable hours.</p>
<p>I highly recommend WakaTime Invoicing to anyone who wants to streamline their invoicing process.
Give it a try, and let us know what you think!</p>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/invoicing"><img src="https://wakatime.com/static/img/blog/invoicing.png" class="img-thumbnail" alt="invoice" style="width:96%" /></a></p>
Alan Hamletthttps://wakatime.com/blog/60-github-adds-wakatime-as-secret-scanning-partnerGitHub adds WakaTime as Secret Scanning Partner2023-02-10T00:00:00Z2023-02-10T00:00:00Z1 min read<p><img src="https://wakatime.com/static/img/blog/leaky-faucet.jpg" class="img-thumbnail" alt="leaky faucet" style="width:90%" /></p>
<div style="font-size:10px;text-align:right;width:90%;margin-top:-18px;margin-bottom:10px;">
Photo by <a rel="nofollow noopener" target="_blank" href="https://unsplash.com/@sanatoga?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Joe Zlomek</a> on <a rel="nofollow noopener" target="_blank" href="https://unsplash.com/photos/zrb_TkHPVtE?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
</div>
<p>We’re proud to announce GitHub has partnered with WakaTime to scan for leaked api keys and other secret WakaTime tokens.</p>
<p>WakaTime tokens allow users to programmatically access their WakaTime code statistics.
GitHub secret scanning protects users by searching repositories for secret tokens, and flagging them for revokation.
By identifying and flagging these secrets, GitHub helps prevent data leaks and fraud.</p>
<p>GitHub will forward access tokens found in public repositories to WakaTime, who will immediately revoke the leaked token and email the token’s owner with instructions on next steps.
You can read more information about WakaTime tokens <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/developers/#authentication">here</a>, and view the original announcement on the <a rel="nofollow noopener" target="_blank" href="https://github.blog/changelog/2023-02-10-wakatime-is-now-a-github-secret-scanning-partner/">GitHub Blog</a>.</p>
Alan Hamletthttps://wakatime.com/blog/59-the-currently-focused-files-expertThe Currently Focused File's Expert2023-02-05T00:00:00Z2023-02-05T00:00:00Z1 min read<p>We’re proud to announce a new feature for our IDE plugins that answers a common dev question:</p>
<blockquote>
<p>Who’s the expert on this code?</p>
</blockquote>
<p>From now on, the name of the team member who has spent the most time coding in the currently focused file will be displayed in your IDE’s status bar.</p>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/teams"><img src="https://wakatime.com/static/img/ScreenShots/org-file-expert-status-bar-screenshot.png" class="img-thumbnail" alt="screenshot" style="width:100%" /></a></p>
<p>Onboarding new hires has just gotten easier.
When a new dev is stuck on some code, they will know exactly who to turn to for questions, saving valuable time and resources.
This feature will also help in fostering a collaborative environment, where team members can learn from one another.</p>
<p>We understand that your devs’ time is valuable, and we want to make their experience as smooth and efficient as possible.
This new feature is just one of many ways we are working to improve the developer experience and make coding that much easier.</p>
<p>This feature was built as a direct result of feedback from one of our users, <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/@be6cf5cc-7f5a-43b3-b789-4a4ddd5b1bc2">Paul Preibisch</a>.
Thank you Paul!</p>
<p>We hope you enjoy it, however if you prefer to disable the file experts in your status bar just add <code>status_bar_team = false</code> to your <code>~/.wakatime.cfg</code> file.</p>
<p>Happy coding!</p>
Alan Hamletthttps://wakatime.com/blog/58-chatgpt-prototyped-our-new-featureChatGPT Prototyped Our New Feature2023-01-24T00:00:00Z2023-01-24T00:00:00Z3 min read<h3>Intro</h3>
<p>I’m naturally a skeptic about OpenAI and ChatGPT, probably because I still think of AI as simple supervised machine learning with markov models.
That’s why it’s taken me a few months to try out the new hotness, <a rel="nofollow noopener" target="_blank" href="https://chat.openai.com/">ChatGPT</a>.
For my first try, I decided to see if it could prototype a new feature:</p>
<p>A bubble chart showing your most used files in a project.</p>
<h3>The Story, according to ChatGPT</h3>
<p>Recently, the WakaTime team was in search of a way to visually represent the amount of time their users were spending on different files.
They knew that a bubble chart would be the perfect solution, but they were struggling with creating one from scratch.
That's when they came across ChatGPT, an AI language model developed by OpenAI.</p>
<p>To their surprise, ChatGPT was able to provide them with a working prototype of the bubble chart feature within minutes.
Using the d3.js library, the prototype generated a packed bubble chart that displayed the time spent on each file as the size of the corresponding bubble.
The team was amazed by the speed and accuracy of ChatGPT's work.</p>
<p>After some minor modifications and hooking the code up to real WakaTime data, the WakaTime team was able to successfully implement the feature on their website.
The new bubble chart feature has received positive feedback from users, who appreciate the visual representation of their file usage data.</p>
<p>The WakaTime team couldn't be happier with the results and are now looking for new ways to utilize ChatGPT's capabilities.
They were impressed by the AI's ability to understand their needs and provide a working solution in such a short amount of time.
This experience has proven to be a game changer for the team and they look forward to more opportunities to work with ChatGPT in the future.</p>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/static/img/blog/openai-bubble-chart-2023-01-24.png"><img src="https://wakatime.com/static/img/blog/openai-bubble-chart-2023-01-24.png" class="img-thumbnail" alt="chatgpt conversation" style="width:100%" /></a></p>
<h3>Results</h3>
<p>Here’s the final code ChatGPT produced, and it works!</p>
<p><img src="https://wakatime.com/static/img/blog/openai-bubble-chart-prototype.png" class="img-thumbnail" alt="output from running the code" style="width:60%" /></p>
<pre><code><!DOCTYPE html>
<html>
<head>
<script src="https://d3js.org/d3.v6.min.js"></script>
<style>
/* set the CSS */
.bubble {
stroke: white;
stroke-width: 2px;
}
.label {
font-size: 12px;
text-anchor: middle;
}
</style>
</head>
<body>
<div id="chart"></div>
<script>
// set the data
var data = [
{ item: "Item 1", time: 20 },
{ item: "Item 2", time: 10 },
{ item: "Item 3", time: 30 },
{ item: "Item 4", time: 15 }
];
// set the dimensions and margins of the chart
var width = 600;
var height = 600;
var margin = 30;
//create a color scale
var color = d3.scaleOrdinal()
.domain(data.map(function(d) { return d.item; }))
.range(d3.schemeCategory10);
// create the chart
var chart = d3.select("#chart")
.append("svg")
.attr("width", width)
.attr("height", height)
.append("g")
.attr("transform", "translate(" + margin + "," + margin + ")");
// set the pack layout
var pack = d3.pack()
.size([width - margin, height - margin])
.padding(2);
// create the root node
var root = d3.hierarchy({ children: data })
.sum(function(d) { return d.time; });
// create the nodes
var nodes = pack(root).leaves();
// create the bubbles
var bubbles = chart.selectAll(".bubble")
.data(nodes)
.enter()
.append("circle")
.classed("bubble", true)
.attr("r", function(d) { return d.r; })
.attr("cx", function(d) { return d.x; })
.attr("cy", function(d) { return d.y; })
.style("fill", function(d) { return color(d.data.item); });
//create the text label
var labels = chart.selectAll(".label")
.data(nodes)
.enter()
.append("text")
.classed("label", true)
.attr("x", function(d) { return d.x; })
.attr("y", function(d) { return d.y; })
.text(function(d) { return d.data.item; });
</script>
</body>
</html>
</code></pre>
<p>After some minor modifications and connecting real data, here’s a screenshot from a <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/projects">WakaTime project dashboard</a>.</p>
<p>And yes, the story section was generated by ChatGPT ;-)</p>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/projects"><img src="https://wakatime.com/static/img/blog/files-bubble-chart.png" class="img-thumbnail" alt="time spent per file as bubble chart" style="width:90%" /></a></p>
Alan Hamletthttps://wakatime.com/blog/57-wakatime-2022-programming-statsWakaTime 2022 Programming Stats2023-01-02T00:00:00Z2023-01-02T00:00:00Z2 min read<p>Happy New Year!</p>
<p><img src="https://wakatime.com/static/img/blog/fireworks-colorado.jpg" class="img-thumbnail" alt="fireworks" style="width:90%" /></p>
<div style="font-size:10px;text-align:right;width:90%;margin-top:-18px;margin-bottom:10px;">
Photo by <a rel="nofollow noopener" target="_blank" href="https://unsplash.com/@jamie_fenn">Jamie Fenn</a> on <a rel="nofollow noopener" target="_blank" href="https://unsplash.com/photos/PEVG_cqrIVo">Unsplash</a>
</div>
<p>In 2022, more than 400k developers spent a combined 43 million hours programming, tracked with <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/plugins">WakaTime plugins</a>.
The average person spent 46 minutes per day programming.</p>
<p>Each year we release a <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/a-look-back-at-2022">personalized coding report</a> showing your accomplishments over the last year.
We also include average trends for all WakaTime devs showing the most used languages, IDEs, and operating systems.
These trends come from aggregating the WakaTime personal dashboard metrics when a developer installs one of our <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/plugins">IDE plugins</a>.
But first, a quick list of new features released in 2022.</p>
<h2>New Features & Milestones</h2>
<h3><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/55-code-insights">📊 Insights dashboard</a></h3>
<p>We built a <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/insights">whole new dashboard</a> for all your code insights.
See which weekday you code the most over the last year, your daily average code time, and more cool insights about the time you spend coding.</p>
<h3><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/me">👩🏼💻 GitHub-style activity table</a></h3>
<p>Add a GitHub-style activity chart to your public profile, showing the frequency of your code time over the last year.</p>
<h3><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/54-custom-project-colors">😎 Custom project colors</a></h3>
<p>Set custom colors for your projects.
All HEX color codes are supported, except for the reserved colors black and white.
That’s over 16 million custom colors for your projects!</p>
<h3><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/52-segment-your-code-time">⚖️ Segment your code time</a></h3>
<p>See your time spent coding vs. attending meetings, or the time you spent coding at home vs. work.</p>
<h2>Programming stats for 2022</h2>
<p><a name="top-languages"></a></p>
<h4>Top languages for 2022</h4>
<p><img src="https://wakatime.com/share/all-users-stats/2022-languages.png" class="img-thumbnail" alt="top languages" style="width:90%" /></p>
<p><a name="top-editors"></a></p>
<h4>Top editors for 2022</h4>
<p><img src="https://wakatime.com/share/all-users-stats/2022-editors.png" class="img-thumbnail" alt="top editors" style="width:90%" /></p>
<p><a name="top-os"></a></p>
<h4>Top operating systems used</h4>
<p><img src="https://wakatime.com/share/all-users-stats/2022-operating_systems.png" class="img-thumbnail" alt="top operating systems" style="width:90%" /></p>
<p>See your personalized report for 2022 here:</p>
<p><a rel="nofollow noopener" target="_blank" href="https://wakatime.com/a-look-back-at-2022" style="
font-size:16px;
font-weight:700;
padding:15px 40px;
color:#fff;
background-color:#2595ff;
border-color:#0b89ff;
text-decoration:none;
display:inline-block;
margin-bottom: 0;
text-align: center;
vertical-align: middle;
touch-action: manipulation;
cursor: pointer;
background-image: none;
border: 1px solid transparent;
white-space: nowrap;
line-height: 1.428571429;
border-radius: 4px;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;">View your personalized 2022 stats</a></p>
<p>These aggregated 2022 stats are <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/api/v1/stats/2022">available as JSON</a>, and also available for <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/blog/tag/yearendreport">previous years</a>.</p>
Alan Hamletthttps://wakatime.com/blog/56-building-a-background-task-queue-in-pythonBuilding a Background Task Queue in Python2022-09-05T00:00:00Z2022-09-05T00:00:00Z3 min read<p><img src="https://wakatime.com/static/img/blog/light-queue.png" class="img-responsive" alt="light queue" /></p>
<h2>Why not just use Celery/RQ/Huey/TaskTiger?</h2>
<p>Unfortunately, WakaTime has been using Celery for almost 10 years now.
During that time I’ve experienced <a rel="nofollow noopener" target="_blank" href="https://github.com/celery/celery/issues?q=author%3Aalanhamlett">many critical bugs</a>, some still open years after being introduced.
Celery used to be pretty good, but feature bloat made the project difficult to maintain.
Also in my opinion, splitting the code into <a rel="nofollow noopener" target="_blank" href="https://docs.celeryq.dev/en/2.5-archived/changelog.html">three separate GitHub repos</a> made the codebase hard to read.</p>
<p>However, the main reason:
<strong>Celery delayed tasks don’t scale.</strong></p>
<p>If you use Celery delayed tasks, as your website grows eventually you’ll start seeing <a rel="nofollow noopener" target="_blank" href="https://github.com/celery/celery/issues/3267">this error message</a>:</p>
<pre><code>QoS: Disabled: prefetch_count exceeds 65535
</code></pre>
<p>When that happens the worker stops processing all tasks, not just delayed ones!
As WakaTime grew, we started running into this bug more frequently.</p>
<p>I tried <a rel="nofollow noopener" target="_blank" href="https://github.com/rq/rq">RQ</a>, <a rel="nofollow noopener" target="_blank" href="https://github.com/coleifer/huey">Huey</a>, and <a rel="nofollow noopener" target="_blank" href="https://github.com/closeio/tasktiger">TaskTiger</a>, but they were missing features and processed tasks slower than Celery.
A background task queue is indispensable for a website like WakaTime, and I was tired of running into bugs.
For that reason, I decided to build the simplest background task queue possible while still providing all the features required by WakaTime.</p>
<h2>Introducing WakaQ</h2>
<p><a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq">WakaQ</a> is a new Python background task queue.
Use it to run code in the background so your website stays fast and snappy, and your users stay happy.</p>
<h2>WakaQ is simple</h2>
<p>It’s only 1,264 lines of code!</p>
<pre><code>$ find . -name '*.py' -not -path "./venv*" | xargs wc -l | grep " total" | awk '{print $1}' | numfmt --grouping
1,264
</code></pre>
<p>It only took <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/commits/main?after=a11d22b6a743e4fb0e220673085297bdc4aab710+104&branch=main&qualified_name=refs%2Fheads%2Fmain">one week</a> from the first line of code until fully replacing Celery at WakaTime.
That says something about it’s simplicity.</p>
<p>Each queue is implemented using a <a rel="nofollow noopener" target="_blank" href="https://redis.io/docs/data-types/lists/">Redis list</a>.
Delayed tasks get their own queues implemented using <a rel="nofollow noopener" target="_blank" href="https://redis.io/docs/data-types/sorted-sets/">Redis sorted sets</a>.
Broadcast tasks share a single <a rel="nofollow noopener" target="_blank" href="https://redis.io/docs/manual/pubsub/">Redis Pub/Sub queue</a>.</p>
<h2>WakaQ has all the necessary features</h2>
<ul>
<li>Queue priorities</li>
<li>Delayed tasks (run tasks after a timedelta eta)</li>
<li>Scheduled cron periodic tasks</li>
<li><a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/blob/761d08f06d7d88941491e48d1cb524a1c35788ad/wakaq/task.py#L47">Broadcast</a> tasks (run a task on all workers)</li>
<li>Task <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/blob/761d08f06d7d88941491e48d1cb524a1c35788ad/wakaq/exceptions.py#L8">soft</a> and <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/blob/761d08f06d7d88941491e48d1cb524a1c35788ad/wakaq/worker.py#L370">hard</a> timeout limits</li>
<li>Optionally retry tasks on soft timeouts</li>
<li>Combats memory leaks by restarting workers when <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/blob/a11d22b6a743e4fb0e220673085297bdc4aab710/wakaq/worker.py#L339">max_mem_percent</a> reached</li>
<li>Super minimal and maintainable</li>
</ul>
<p>Features considered out of scope are rate limiting, exclusive locking, storing task results, and task chaining.
Those are easy to add in your application’s task code, and you probably want to implement these specific to your app’s needs anyway.</p>
<h2>WakaQ is ready to use</h2>
<p><a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq">WakaQ</a> currently powers all background tasks in prod for the <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/">WakaTime</a> website, including but not limited to:</p>
<ul>
<li>sending code stats email reports</li>
<li>renewing our LetsEncrypt SSL certs</li>
<li>pre caching dashboards, repo badges, and embeddable charts</li>
<li>anything else we don’t want holding up the web requests</li>
</ul>
<p>It’s released under a <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/blob/main/LICENSE">BSD license</a>, so you can use it in open and closed source projects.
If you find any bugs please <a rel="nofollow noopener" target="_blank" href="https://github.com/wakatime/wakaq/issues">open an issue</a>, but think twice before requesting new features :-)</p>
<p>Happy coding!</p>
<p>- <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/@alan">Alan</a> from <a rel="nofollow noopener" target="_blank" href="https://wakatime.com/">WakaTime</a></p>
Alan Hamlett