How I am learning to organize files

I’ve never been a disorganized person but it’s difficult to organize files in a clean way. Files are created which fit into multiple folders and now you don’t know where to look. I looked on reddit to see if anyone has found a good method for naming folders.

I found a suprisingly active niche community /r/datacurator which is dedicated to this simple question: How can we organize files in such a way that everything is easy to find and that we don’t spend a lot of time looking for them?

One of the posts on the subreddit linked to “Johnny.Decimals”. The idea is that you can use numbers to organize your folders in a certain order such that “Writing” can be near “Documents” even though they won’t be with alphanumeric sorting. You give each group a common number and organize categories into “area folders”.

At first this seemed like a silly idea. I want less folders, not more. But I can take a few good ideas and create a system which works for me.

First, I want to emphasize that my solution will not be good without fast search methods. If I’m looking for a specific document then I will rely on tools like fd or Everything (one of the only benefits of using Windows/NTFS).

I plan to store my files only one or two folder levels deep. I want clean categorization without having to dig through nested folders.

I already have very generic sorted folders but one of the problems that I’m having is that source data (music samples) are far from my projects. Another problem is that my Sync folder (syncthing, connected to my phone and laptop) has basically the same duplicated structure. Instead of having one folder for source data or syncing I’d rather have many folders if that gave me the cognitive benefit of having data of the same media types closer together.

The first digit will denote the media type and the last digit will denote the function. For example, Pictures that I want to sync between my phone, laptop, and desktop will be in the folder named “40 Pictures”. Stock photos will be in “47 Stock Photos”. My photography .RAW/CR2 files will be in “48 Raw” or “497 Photography Raws”.

Text documents will live in folders starting with `5`.

Synced Documents? `50`

Wikipedia Downloads? `57`

Wikimedia Downloads? `47`

Large InDesign files? `59`

eBooks? `55`

The numbers allow you to put all your folders into one big folder without having all your categories disorganized. The number-prefix will make it such that everything is sorted correctly.

You don’t have to do it the way that I’m doing it. For example, you could turn this unordered list…

  • license

  • bills

  • demos

  • proposals

  • flipcharts

  • data

  • screenshots

  • travel

  • templates

  • contracts

  • processes

Into something that looks more like a pipeline:


  • 1* Relationships

  • 4* Org

  • 5* Product

  • 8* Sales

Status Types:

  • *5* Draft

  • *9* Final

Access Types:

  • *1 Public

  • *2 Restricted

  • *3 Confidential

Potential Result:

  • 11 Contracts

  • 12 Proposals

  • 41 Licenses

  • 41 Processes

  • 42 Templates

  • 42 Bills

  • 43 Travel

  • 51 Demo

  • 51 Screenshots

  • 52 Data

  • 81 Flipcharts

Cities of the World

Curating a list of every major population center

I’ve thought about visiting every city on Earth but it’s kind of an impossible task. You’d ideally need at least three days per city so that you don’t become completely worn out. But 3 days * 7000 cities is already 57 years.

Online, I can virtually visit ten cities in an hour if I rush it. That’s one month of 24/7 city watching. Maybe something more like 3 per hour is more reasonable and it is kind of exhausting to spend the whole day doing that so maybe 2 hours per day. So about 3 years to visit every city on Earth. Not bad.

And so that’s what I’ve started doing from Jul 23 2020. 156 days. I only have about 300 videos total so far but I haven’t been doing this every day. So maybe it will take me about 6 ~ 9 years to complete.

I’ve sorted my list of cities in a spatial way. Using GIS data I’ve created an ordered list of cities and from there I categorized it into five partitions.

Ideally a concise city tour video but often slow TV / walking videos:

If you don’t have a lot of time then you can watch this playlist: 42 Unusual Cities

If anyone wants to help find videos let me know

Sunsets forever

How to make a website in 60 mins

So it was thursday night and I was snarfing the web as usual and I came across an interesting github repo: dantaeyoung’s continuous-sunset. A 2018 vintage. truly exceptional. ripened perfectly. OK so I’m digging through these files and eventually I surmise the premise of the website: you go to the website during any time and it will show you a live video of a sunset somewhere in the world. I think it’s a cool idea and one that I could implement in a few hours. It didn’t look like dantaeyoung made very much progress. It looks like he is tackling the problem a little too precisely.

Sunsets are big things. The pattern is simple: every 24 hours sun goes down. Sure it changes with the seasons but I think for the first iteration of this I will choose to not calculate the exact azimuth of the sun. Sunsets last about 40 mins. So we only need about 24/(40/60) = 35 locations for an MVP website. Will 35 be enough? Maybe. We don’t want the sun to actually set—then it wouldn’t be eternal sunset. We are trying to create limbo here. For the MVP I will consider it a success if the sun doesn’t set but we might see the sun be too high in the sky to be considered “sunset mode” by victorian-era scholars.

Let’s build!

Despite what you have been taught, the first step of building is planning. Any time spent planning will generally give you 5x reduction of building time.

To make sure we are getting the right data we should first think about what kind of data we would need to make this website then we should think about how we should structure this data (data types).

What data do we need?

The answer might not seem simple to you. I think this skill comes with practice. For this project I think we will be fine using only two pieces of data: 1) the current UTC time 2) a list of videos streams to play

Streaming video used to be a royal pain in the butt; YouTube changed that.

How should we store this data?

For later iterations it would be nice too add some variation to the site. 400 locations would not be too much work to add. We won’t do that right now—good heavens, no! But planning ahead will allow us to easily add features in the future. So we need some kind of list. If we had a big list then what we could do is just filter out which ones don’t have a sunset at the current time and then choose randomly from that subset.

For the big list I think an array of objects would be an appropriate choice. For an MVP with only 35 locations we could probably get by with only a string array of youtube links but it would probably be nice to include some metadata about each city like the pre-calculated sunset start and sunset end times (using SunCalc or Wolfram Alpha). We can also include the geospatial coordinates of the city so that if we find that we need to calculate the azimuth in a later version all the data will already be there for us. All of this city data is already in this very high quality dataset so we just need to figure out how to convert it into array of objects format for our website.

How do we open this file? The files inside might seem daunting at first but we don’t even need to unzip it. Just download QGIS and drag-drop the file into a new project.

Okay so we need some way to subset this data to choose 35 good locations that are near the equator so that we don’t have to account for the Earth’s wobble in our MVP.

For now I’m just going to choose Vector->Research->Select Random within Extent, Select Extent on Canvas: I’m limiting it to near the equator so the sunrise and sunset is not too wonky, number of points: 35, minimum distance between points: 8 (because we want to have enough coverage of Earth to have the sun set all the time!). This will create a bunch of random points within a square.

Then I’m going to run Distance to nearest Hub which will calculate the closest random point for every city.

Next we will save it into a GeoJSON file and only save the name, iso_a2, latitude, longitude, ne_id, and HubName (the nearest random point). Because it is GeoJSON we actually don’t need to keep the latitude and longitude attributes since those are saved in the file already.

GeoJSON has a slightly different structure than our initial design plan but it is probably a better format for this kind of data anyway.

Now we should select only one city for each HubName. And we may as well choose the city closest to the equator for each HubName. Unfortunetly, it looks like Distance to nearest Hub is not doing quite what we want here since there are only 22 results. What we really needed was a One-to-One spatial join but Hub is doing One-to-Many. Ohh I think what we should have used was NNJoin but for now we’ll just work with this and we can update the data in the next iteration of our website (after Series A funding of our sunset website).

So I think the logical way to do this would be to create a list of ne_id and the sunrise start and end. Then filter out the list of all places to only include cities where sunset has started and choose the one where the time between now and sunsetEnd is maximized. Then we will check every five minutes to see if it should switch YouTube links. (For the “swap out the video” part I’m not sure exactly how to do that but I remember having a similar style of video swapping.)

Since we aren’t likely to compare across days I think using minutes in the day to compare times will be okay…

Looking at actual sunsets; maybe my initial reaction was a little niave. Sunsets can be short. Maybe 4~20 mins is a better estimate. Also, looking at a cloudy sunset is a bit anti-climactic. I think we might also need to use a weather API and choose the location which has the clearest skies from our list of available live sunsets.

Maybe for an MVP we can just have a link to a youtube search of city live streams which should have a live sunset. Originally I was only planning a simple Alpine.js website but I think we may need to use the YouTube API in future iterations. To keep costs under control we will probably need a backend server so I’m going to set up Elixir LiveView to host our MVP. That will allow us to easily add features like automated YouTube searching for our little website.

edit: this post was never finished because it turns out there aren’t as many live video feeds around the world as I thought

Forgotten Ideas: Twitter, Oct 2007


These are always sorted by most novel so if the first one isn’t interesting you can just delete the email.

This is a really great video. I never knew this history. Highly recommended.

Imagine that when you shopped online for a digital camera, you could see whether anyone you knew already owned it and ask them what they thought.

I feel like Facebook hasn’t really done this yet.

Imagine that when you searched for a concert ticket you could learn if friends were headed to the same show.

But this is implemented in Facebook and it works fairly well.

Or that you knew which sites - or what news stories - people you trust found useful and which they disliked.

This kind of exists but generally Facebook friends don’t have very much information like that. There isn’t a good interface for accessing that information. Facebook is kind of like a locked box in some sense. Hard to query information—even your own info.

Or maybe you could find out where all your friends and relatives are, right now (at least those who want to be found). We want to make Facebook into something of an operating system so you can run full applications
Max Levchin, who co-founded PayPal, is CEO of Slide, which is putting its slide-show service on the new platform. "These guys are creating the opportunity to build Adobes and Electronic Arts and Intuits that live within Facebook," he says.

Facebook-as-a-platform has largely failed. Farmville might be the only true successful Facebook platform application. I think the biggest reason why all these things failed is because Facebook is largely a closed platform that makes money from your data.

Sometimes metaphors and jokes lead to good ideas.

Theses ideas are perhaps lost to the harsh winds of time:

Forgotten Ideas: Twitter, Aug-Sept 2007

De-centering power

I like using Twitter for ideas because the best ideas usually happen by accident. Usually by mishearing someone or misunderstanding what their actual point is. Here is one example. What does user-created company mean? It could mean anything really. It points to decentralization but there are a million ways a company could qualify as “user created”.

If you read their business plan, “user created” becomes more clear, but if, before reading, you take a moment and try to brainstorm what one new phrase like this could mean then you might come up with genuine new thoughts which haven’t been explored. Maybe not on your first try but I think it is good practice to often try.

BTW if you are interested in decentralization + tech then this subreddit is one of the only places I have seen serious philosophical / realistic technical discussion.

Jottit makes getting a website as easy as filling out a textbox

There are lots of tools for this nowadays but back in 2007 this was still a pretty new idea: make website creation more accessible so anyone can do it. Facebook Pages and Timelines are just a variation on that idea. “Makes posting as easy as filling in a box.”

Jottit was one of the many things made by Aaron Swartz. You can watch a free documentary about him here. The website is still up (although it is very slow). I discovered the original website code—and his advice about starting a new project.

This one is kinda funny.

The idea that if you wait long enough, someone will write/build/design what you were thinking about.

Their website is a “coming soon” page… maybe a little on the nose.

Okay so I’ve seen this product mentioned a lot but I’ve been skipping it because I thought it was (the blog site) but it is not.

Social discovery in real time

Me.dium Changes Name To OneRiot, Launches Social Search Engine

Walmart acquires OneRiot, search-startup-turned-ad-network

Loading more posts…