software

Standard Notes is a Better Project than Braindump

released braindump to the world last year to much fanfare. After the initial excitement from being on HN died down, and the PR’s stopped rolling in, it became a personal project once again with very few users. Over the last few weeks I have made several attempts to fix the spaghetti mess that is the current code base by refactoring the current Flask implementation, then rewriting it completely in Django, and even started a branch to investigate rewriting the whole app in PHP using Laravel. Other commitments took precedence and Braindump remains in a fairly usable but not that special state.

Today on HN I read about a new project called Standard Notes which is the most exciting note related project that I have seen in a long time. It solves so many of the problems around cross platform compatibility that plague many other note tools. In addition its goals are to create a standard file format for simple, secure, and durable notes. Even more it has already created a platform, an ecosystem, that allows anyone to come and create additional applications, plugins, and use cases for notes.

These are some of the problems that I set out to tackle when I started braindump. After reading about Standard Notes, and using it for a few hours, I have decided that my time would be better spent contributing to that project instead of continuing to work on Braindump.

Working on Braindump has been amazing. I learned a ton, became a better programmer, and most of all had a lot of fun. I want to thank everyone who tried it, provided feedback, and sent patches. The source code for braindump will remain on GitHub but I would encourage you to try and contribute to the Standard Notes project along with me.

Standard
programming

That UI Bug with Missing Data is a Security Issue

This is (sometimes) a development blog, so I am going to write about some failed development of mine since writing about success is much less interesting. You know that UI bug that someone added to your GitHub Issues where there is some missing data? You know the one, it only happens in production, all of your tests pass, and you marked it as a low priority. Yeah, that one. It’s probably a security bug and you should look into it right away. At least, that is the lesson I taught  myself yet again when I began to research this bugBefore

<a id="shared_note.id)" href=""{{">
{{current_user.notes.filter_by(id=shared_note.id).first().title}}
</a>

After

<a href="{{ url_for('main.note', id=shared_note.note_id) }}">
{{current_user.notes.filter_by(id=shared_note.note_id).first().title}}
</a>

The difference is very subtle, but the key issue here is shared_note.idvs shared_note.note_id; I released a feature a few weeks ago that showed you all of the notes that you have shared. Locally everything worked fine, but I noticed later on, once it was in production, that the note title was not showing up. This is, of course, due to the fact that rather than showing the title of the note with the ID shared_note.note_id (the foreign key linking to the note) I was showing the title for the note with the primary key of shared_note.id. The reason why this is a security issue is because this allows someone to share a bunch of notes and start seeing the titles for notes that they do not own. The reason why this worked locally is because I am only testing with a single user, with a single notebook, with a single note, and with a single shared note. This means that in this specific case all of the Primary Keys and Foreign Keys are usually “1” so everything just happens to work.

Key Takeaways

  • Always test with multiple users, make your local environment as similar to production as possible
  • Consider using UUID instead of Auto Incrementing Integers, this would have been immediately caught if that was the case.
  • “Partial Missing Data” == Security Bug (most of the time)
Standard
programming

Version 0.4.3 of Braindump, I Need More Functional Tests.

Yesterday, I shipped a new release of braindump. It was awesome. Until I tried to use it this morning and realized that notes were no longer being saved when you edit them. So today I cut a new release that fixes that bug, as well as implementing a new feature where you can see what notes you have shared in the past. This also addresses an issue with sharing notes, so you should be able to send notes via email again. Normally, when a product is being released on a regular cadence it is exciting. In this case, it just means that I really need more functional tests.

Post Mortem of Note Saving Bug

For those who are curious, I will document what happened in the hopes that it does not affect others in the future. Braindump was originally written using WTForms, which has CSRF protection built in so you don’t have to think about it. Recently, I have begun transitioning away from WTForms in lieu of regular forms that are submitted via AJAX. The most visible benefit is that you can see and edit all of your notes from the main page, instead of having to click into each note in order to edit it. After switching to AJAX, I quickly realized that I was no longer sending a CSRF token with POST and PUT requests, which is probably a bad thing. I enabled it in the previous release. Even though I got this to work in the login and registration modal (which are true forms), I didn’t consider the PUT request when editing a note. This is not a form at all, but a method that fires whenever you stop typing. Naturally, since there was no token and the endpoint was not marked as exempt, the response was a 400 error (missing CSRF token). My solution was to add the token as a meta tag to every page, and then send it along in the request headers of the AJAX call.

Enable CSRF

// app/__init__.py

csrf = CsrfProtect()

def create_app(config_name):
    ...
    csrf.init_app(app)
    ...

    return app

Token in the Header

// app/templates/app/app_base.html
// This works because csrf_token() is in the global scope.

<meta name="csrf-token" content="  csrf_token() " />

AJAX Method to add the token before sending the PUT request.

// frontend/js/src/braindump.editor.js

saveNote: function(id, content) {

        var csrftoken = $('meta[name=csrf-token]').attr('content')

        $.ajax({
                beforeSend: function(xhr) {
                        xhr.setRequestHeader("X-CSRFToken", csrftoken)
                },
                url: `/edit/${id}`,
                data: JSON.stringify({
                        'body': content,
                }),
                contentType: 'application/json',
                type: 'PUT',
                success: function (response) {
                        console.log(response);
                },
                error: function (error) {
                        console.error(error);
                }
        });
},

Everything now works fine, and in the future if I need to make more AJAX calls, the token will be available in the meta of each page. This would have been easily caught if I had had even a single functional test. I hope to have something in place over the next few days so that I stop shipping breaking changes. Braindump is still very beta level software. Once we make it past the 1.0 release, I will have close to 100% code coverage, full functional test suite, and develop a steady pace for incremental improvements so that there are no more surprises like this one in the future.

Standard
programming

How to Ship a Simple Feature in Two Months

I started working on a feature to allow users to remove notebooks in Braindump two months ago. What started as adding an additional link and a corresponding flask endpoint to handle the deletion has turned into a horrible long running branch that I have no idea how to get out of. This is a common trap that I fall into, and I have seen other developers struggle with the same issues.

Scope

Add a Feature to allow users to remove notebooks.

Implementation

Easy Peasy, here is a link to delete notebooks. Ship it.

Side Tracking

Well, while we are here, maybe we should use the new “card” style that is hip now. Hmm, wait, this looks stupid in purple. Let’s make it blue. The cards look great, lets make the rest of the site blue as well. Ok it looks awesome in blue. Have not looked at CSS in a while, maybe we should refactor it, switch to SASS and concatenate all of the CSS into a single file. Oh crap, doing this breaks the outer theme compared to the inner theme since they were defining rules for the same elements. No problem, I will just rewrite the whole UI again.

30 commits later

What problem was I trying to solve again?

Conclusion

This is actually what happened. No matter how many times I tell myself I will never fall into this trap again, I keep on doing it. Next time will be different. In any case, the new notebook view looks good and allows you to delete notebooks. I hope to have this shipped this year.

Standard
programming

Braindump Versions, Release Notes, and the Road Ahead

I have been playing with a couple different methods of versioning and release notes with Braindump. I used to do all release notes in GitHub. GitHub “Releases” are nothing more than git tags with some additional meta-data and after talking to eric one day I agree that locking myself into GitHub specific tags is not the best idea. Then I started to add release notes on my blog and just link to it from the GitHub tag, this worked ok but you may have noticed that I blogged about version 0.3.0 yesterday and then released versions 0.3.1, 0.3.2, 0.3.3, and 0.3.4 today. I think I finally came up with a solution that makes the most sense to me.

Braindump is using semver, so the scheme is MAJOR.MINOR.PATCH. So going forward, I will create release notes only for MAJOR and MINOR versions.

With the new Docker deployments I am creating two types of containers.

  1. latest which refers to the latest commit on master
  2. stable which refers to the latest git tag

I also have created some milestones and put every current issue into a milestone. Patch versions are now milestones and they will come and go quickly. Major and Minor versions are going to be more long running milestones. The Ice Box is where dreams to go ~die~ come true some day.

Braindump.pw will always be running the latest patch version, or stable. I hope this new methodology will provide some better structure and clarity for this project.

Standard
programming

Version 0.3.0 of Braindump Released

I am excited to announce the release of version 0.3.0 of Braindump. This release contains few new features, but some pretty big changes have happened with the way that Braindump is being deployed.

New Features

  • Switched to using Prose Mirror instead of ACE. I have been super impressed with Prose Mirror and I think that it will make the writing experience on Braindump much better.

UX

  • New Logo (thank you Songbing :))

Ops and Dev

  • Switched to NPM for Javascript Dependency Management
  • Completely Dockerized the application using Docker Compose, CircleCI and Quay.io, each git tag is now being automatically deployed to production while each master build is building an image.
  • Completely Switched to Python 3 (3.5 to be exact). Props to Ricardo for always trolling me about living in the past.
Standard
programming

Version 0.2.6 of Braindump Released

I am excited to announce the release of version 0.2.6 of Braindump. I have taken a short break from Braindump at the beginning of the year, but I am still committed to making it the best note platform out there!

New Features

  • Note Archives
  • Ability to Completely Empty the Trash

Removed Features

  • Oauth[1]
  • Tasks[2]

UX

  • Note is kept active once it has been updated
  • Return to Same Notebook when adding a new note from within a Notebook

[1] Oauth has not been working for a while, and I want to dig into it more so I have removed it for now.

[2] Braindump is a Note tool first, I reworked Tasks in a previous release, but decided to remove it completely for now. I want to focus on making the best possible Note Platform before moving on to additional functionality.

I am planning on updating the docs to make it easier for people to contribute. In addition, I am planning yet another UI overhaul. I want to make the app simpler and more appealing.

Standard
programming

Version 0.2.5 of Braindump Released

I am excited to announce the release of version 0.2.5 of Braindump. It has been a while since I have made a new release, but there have been enough new features added over the last few weeks that I felt that it was time! One of the most exciting things about this release is the addition of GitHub OAuth, and full TODO functionality. One of my primary motivations for creating Braindump was to offer a platform for taking actionable meeting notes. We are now pretty close to achieving that goal thanks to the addition of TODOs. You can add TODOs inside a note, mark them as completed, and also view all of your TODOs across many notes. This allows you to keep your notes short and focused while not having to worry about missing a TODO item from some note that you wrote weeks ago.

I am continuously inspired by the response, feedback, and participation that Braindump has received over the last few weeks. The next few releases are going to focus on polishing the UI, making the API more robust, and make it easier to create native apps. My goal for Q1 2016 is to be able to have a native braindump client app available for mobile and desktop.

You can read the full release notes here and you can always use the latest version of braindump on the official site.

Standard
programming

I Shared Braindump With the World

Yesterday I wrote about my new app called braindump that I have been working on. Today I added a couple new features and decided to share it on Hacker News. I was only expecting a few people to check it out, and I have been thrilled with the response and feedback.

  • 259 Stars on GitHub
  • 89 Points on HN
  • 192 New users on braindump.pw
  • Front Page!

Thank you to everyone who checked out braindump and provided some great feedback. I am going to take all of the feedback that I have received so far and keep it in mind as I continue to develop this app. I am planning on adding a bunch of great new features as well as begin working on an API to make braindump the “notes platform” that it is promised to be rather than just another note app.

This is my first “real” project, and also my first time sharing a project like this on HN. In hindsight I probably could have waited a bit to share it with the whole world since it is still in the early stages of development. However, I could not be happier with the response. Thank you to everyone who looked at it, I hope you will continue to use my little app and provide feedback.

Standard
programming

Introducing Braindump

I have been working on braindump for over a year now. I have rewritten it from scratch over 10 times. Three weeks ago I finally wrote a version that I am somewhat happy with and I am excited to share it with the world.

I have a vision for what a notes app/platform is supposed to look like and unfortunately I have not been able to find something that fits that vision. This is my attempt at making something myself that fulfills that vision.

I have been using it myself for a few weeks at work and it is really cool to be able to use software that you wrote yourself. This latest release has focused on making the UI a bit better. I am proud of the work I have done and would love to hear your feedback.

This release also included some commits from other developers which is awesome.

The project is live here, I hope you will give it a try and let me know what you think!

Standard