How to Crawl the Web with Scrapy
How to use the python web scraping framework Scrapy to crawl indeed.com. Learn data engineering strategies for getting actionable insights from public information.
Every blog post that took off on Hacker News, ranked by the votes of its best submission. 80,999 posts.
How to use the python web scraping framework Scrapy to crawl indeed.com. Learn data engineering strategies for getting actionable insights from public information.
There’s a concept from computer security known as a class break. It’s a particular security vulnerability that breaks not just one system, but an entire class of systems. Examples might be a vulnerability in a particular operating system that allows an attacker to take remote control of every computer that runs on…
Learn what vector search is and the metrics pertinent to decide the distance (or similarity) between objects.
Ever wanted to know how exactly did Quake’s precomputed visibility work? I did, so I wrote vis.py , a reimplementation of their algorithm in Python. This guide has all the information you need to understand vis , the tool used by Quake, Half-Life and Source Engine games.
No single cloud fits all. Explore how we build a more flexible future with hybrid cloud.
Some time ago, I came across the Wikipedia article of Niels Henrik Abel , and something there burnt its way into my mind. Abel changed the face of mathematics, despite dying at 27. And here comes the thing – I give the mic to Wikipedia now:
Today we’re unveiling our latest moonshot from Google[x].
We have been experimenting with a couple of approaches to bring Mono to the web using WebAssembly - a technology that can efficiently and safely execute code in web browsers without being limited to Javascript. Running code written in C or C++ inside the browser has been a big motivator, but most major programming…
Wayland is all the rage those days. Distributions left and right switch to it, many readers of my previous article on writing a X11 GUI from scratch in x86_64 assembly asked for a follow-up article about Wayland, and I now run Wayland on my desktop. So here we go, let's write a (very simple) GUI program with Wayland…
A core value CloudFlare is that security information should be shared between organizations to make the entire Internet safer. That is how CloudFlare's systems work: if one site is attacked, data about that attack is immediately shared with the rest of the network so other sites can be safe.
Only once in a while does an Internet censorship law or regulation come along that is so audacious in its scope, so misguided in its premises, and so poorly thought out in its execution, that you have to check your calendar to make sure April 1 hasn't come around again. The Draft Online Regulation...
Varnish is a powerful, flexible HTTP accelerator that speeds up web applications by caching content and optimizing delivery.
Although the presence of ET planetary-based life as we know life and don’t know it, is all, comparatively as, and less and in gradations more technologically and cognitively achieved or advanced than our species, such life, in mathematical and thermochemical terms, is a must on a vast in raw numbers but quite possibly…
You can make your own website. This guide can help anyone make a website with HTML and CSS. You can make a personal site, link tree, hobby site, digital garden, fansite, or any other website you want. It's fun, and easy enough for a 12-year-old.
I recently had solar panels and a battery storage system from GivEnergy installed at my house. A major selling point for me was that they have a local network API which can be used to monitor and…
You're probably reading this page because you've attempted to access some part of my blog (Wandering Thoughts) or CSpace , the wiki thing it's part of. Unfortunately you're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old (most often this applies to versions…
Digital platforms play an ever-increasing role in structuring and influencing public debate. Civil society watchdogs, researchers and journalists need to be able to hold them to account. But Facebook is increasingly fighting those who try. It shut down New York University’s Ad Observatory last week, and went after…
Exploratory interfaces for navigating photos
News and updates from the RubyGems team — releases, security advisories, and project announcements.
One day in January 2013 I found myself wasting time on the internet.
Developed by a team of Harvard researchers, the first autonomous, entirely soft robot is powered by a chemical reaction controlled by microfluidics. The 3-D-printed “octobot” has no electronics.
The Internet privacy company that empowers you to seamlessly take control of your personal information online, without any tradeoffs.