Every blog post that took off on Hacker News, ranked by the votes of its best submission. 1,888 posts.
While Full-text-search is famous for search through documents at a faster rate, there are other interesting ways in which these systems handle scale, synonyms, abbreviations, misspellings etc., we will
Kristoffer over at rpsychologist.com wrote a really nice tutorial on getting up to speed with maps in R. It looked something like this.
CQRS (Command Query Responsibility Segregation) is the notion that you can use a different model to update information than the model you use to read information
Recapping the criticisms made about MongoDB.
Databases shed important RDMS features as they get big. Examining why this tends to be the case, and some ideas for preventing it.
I recommend using Python for data analysis, and I recommend Wes McKinney's book Python for Data Analysis. I prefer Python to R for mathematical computing because mathematical computing doesn't exist in a vacuum; there's always other stuff to do. I find doing mathematical programming in a general-purpose language is…
PostgreSQL always allowed, by default, any user to connect to any database, and create new tables (and views, and so on) in “public" schema. The one that is created by default.
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
Explore the fundamentals of the DICOM file format! This quick introduction covers the basics of DICOM's structure, its essential uses, and tips for easily navigating its complex and abstract components.
I often work with clients who have large “data lakes” or big star schema style enterprise databases with fact and dimension tables as far as the eye can see. Invariably said clients end up with a substantial SQL codebase composed of hundreds of independent queries with lots of overlap between them. I want to be able…
Learn how to load larger-than-memory NumPy arrays from disk using either mmap() (using numpy.memmap), or the very similar Zarr and HDF5 file formats.
The Django ORM feels so much easier than SQLAlchemy? Maybe this article can change your perception.
Once a month users post a chart on `r/ethereum` predicting the blockchain size of Ethereum will soon exceed 1 TB. I want to clean up with some stories around the Ethereum-blockchain size in this article and try to explain why this chart is technically correct, but not the full picture...
Comparing the genome sequences of the Wuhan coronavirus and a bat coronavirus using `diff` and NCBI BLAST reveals an 89% similarity.
One of the easiest ways to start visualizing data is to turn a table into a heatmap: every cell gets a colour, the higher the number the brighter the colour. Unfortunately, this is often a fairly unrewarding exercise, yielding graphics that look like plaid or tartan fabric. Part of the problem is that the rows and…
Last week, a blog post by an Uber engineer explained why Uber chose to move from PostgreSQL to MySQL . This article was widely reported and...
I recently had to set up a new postgres instance at work. While we’re already using Heroku Postgres for production loads, we had some requirements for this one that couldn’t be solved without being database admin, and thus had to go self-hosted. That...
MySQL and MongoDB are both popular, yet poor representatives of their species
I built a new plugin for my sqlite-utils CLI tool that lets you ask human-language questions directly of SQLite databases and CSV/JSON files on your computer. It’s called sqlite-utils-ask. Here’s …
An interesting challenge in cartography is getting data of the appropriate resolution. As Lewis Fry Richardson observed , the more precisely you measure the length of a coastline, the longer it appears; geographic shapes have infinite complexity. Digital displays, in contrast, are limited by pixels. Choosing too high…
This is Jessica. In my Prediction for Decision-making class last quarter we talked about human predictions and decisions relative to algorithmic ones. This led to a discussion about why it’s often hard to demonstrate the value of human knowledge once you have a decent statistical model.
This post shows how to process gigabytes of unstructured data with the Spark framework and using the R language.
