You also don't -have- to sign in. So your post is just nonsense."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Why Write Python in Visual Studio?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://blogs.msdn.com/b/visualstudio/archive/2015/08/03/why-write-python-in-visual-studio.aspx"}},"_tags":["comment","author_partiallypro","story_9998227"],"author":"partiallypro","children":[9999770,10001446,10017885],"comment_text":"You mean the "keylogger" than only tracks search queries & auto-correct to better optimize algorithms, which Google and Apple both do. Sure.
You also don't -have- to sign in. So your post is just nonsense.","created_at":"2015-08-03T20:26:19Z","created_at_i":1438633579,"objectID":"9999519","parent_id":9999175,"points":null,"story_id":9998227,"story_title":"Why Write Python in Visual Studio?","story_url":"http://blogs.msdn.com/b/visualstudio/archive/2015/08/03/why-write-python-in-visual-studio.aspx","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"coffeemug"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"RethinkDB - www.rethinkdb.com - Mountain View, CA - C++ hackers
We're making it dramatically easier for the world to shift to realtime apps with a new database access model -- instead of polling the database for changes, the developer can tell RethinkDB to continuously push updated query results to applications in realtime.
We're hiring C++ engineers to work on performance, the continuous computation engine, and the distributed system.
RethinkDB is fun, very technically challenging, well-funded, and is growing very quickly. Intellectually, we're also PL nerds (mostly Lisp and Haskell/ML), so if that's your cup of tea you'll be right at home!"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_coffeemug","story_9996333"],"author":"coffeemug","children":[9999353,10002528,10003257,10004817,10006573,10010575,10018926,10075061,10075063],"comment_text":"RethinkDB - www.rethinkdb.com - Mountain View, CA - C++ hackers
We're making it dramatically easier for the world to shift to realtime apps with a new database access model -- instead of polling the database for changes, the developer can tell RethinkDB to continuously push updated query results to applications in realtime.
We're hiring C++ engineers to work on performance, the continuous computation engine, and the distributed system.
RethinkDB is fun, very technically challenging, well-funded, and is growing very quickly. Intellectually, we're also PL nerds (mostly Lisp and Haskell/ML), so if that's your cup of tea you'll be right at home!","created_at":"2015-08-03T19:50:13Z","created_at_i":1438631413,"objectID":"9999278","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mjwhansen"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"The Motley Fool (Alexandria, VA (outside DC)) -- Glassdoor's Best Small or Medium Sized Place to Work for 2 years running -- is looking for a full-time, onsite DBA.
Core Competencies\n-Ability to document new processes and changes to the database environment\n-Experience configuring, administering, or developing for MS SQL Server 2008/2012\n- Excellent SQL skills and understanding of triggers, stored procedures, and indexed views\n- Ability to diagnose and triage production issues without supervision\n- Ability to work directly with internal customers / project owners\n- Experience with performance tuning and ability to find and tune query- or server-scoped performance bottlenecks\n- Test and release changes to databases in the context of a software release lifecycle\n- Ruthless prioritization skills \u2013 you know the different between \u201curgent\u201d and \u201cURGENT,\u201d and those last-minute, behemoth requests don\u2019t even make you blink\n- Equally strong commitment to leaving things better than you found them AND leaving a lasting, positive impression\n- You must love data!
Considered A Plus:\n- Knowledge of other databases and analytical datastores\n- Experience working with Virtualization technologies (VMware, Hyper-V)\n- Experience using Python for data processing and/or pipelining
http://hire.jobvite.com/CompanyJobs/Careers.aspx?k=Job&j=o9w..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_mjwhansen","story_9996333"],"author":"mjwhansen","comment_text":"The Motley Fool (Alexandria, VA (outside DC)) -- Glassdoor's Best Small or Medium Sized Place to Work for 2 years running -- is looking for a full-time, onsite DBA.
Core Competencies\n-Ability to document new processes and changes to the database environment\n-Experience configuring, administering, or developing for MS SQL Server 2008/2012\n- Excellent SQL skills and understanding of triggers, stored procedures, and indexed views\n- Ability to diagnose and triage production issues without supervision\n- Ability to work directly with internal customers / project owners\n- Experience with performance tuning and ability to find and tune query- or server-scoped performance bottlenecks\n- Test and release changes to databases in the context of a software release lifecycle\n- Ruthless prioritization skills \u2013 you know the different between \u201curgent\u201d and \u201cURGENT,\u201d and those last-minute, behemoth requests don\u2019t even make you blink\n- Equally strong commitment to leaving things better than you found them AND leaving a lasting, positive impression\n- You must love data!
Considered A Plus:\n- Knowledge of other databases and analytical datastores\n- Experience working with Virtualization technologies (VMware, Hyper-V)\n- Experience using Python for data processing and/or pipelining
http://hire.jobvite.com/CompanyJobs/Careers.aspx?k=Job&j=o9w...","created_at":"2015-08-03T19:47:48Z","created_at_i":1438631268,"objectID":"9999260","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gibrown"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"REMOTE - Full Time - Data Wrangler - Automattic (makers of WordPress.com, Jetpack, Polldaddy, Gravatar)
We're a distributed company with employees in 42 countries. Help us influence the 23% (oops now 24%) of the web that runs on WordPress.
We're building out our data infrastructure. Each day we handle:
- 3.5 Million New Posts & Comments From 196 countries
- 39 Million Elasticsearch Queries In 137 languages
- 33 Terabytes of Elasticsearch Data for 175M+ Unique Visitors
- 0.5 billion pageviews
- Millions of events streaming through Kafka, Hive, Impala, eventually Spark
No walls around the garden. Make the Open Web a smarter place.
http://automattic.com/work-with-us/data-wrangler/"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_gibrown","story_9996333"],"author":"gibrown","comment_text":"REMOTE - Full Time - Data Wrangler - Automattic (makers of WordPress.com, Jetpack, Polldaddy, Gravatar)
We're a distributed company with employees in 42 countries. Help us influence the 23% (oops now 24%) of the web that runs on WordPress.
We're building out our data infrastructure. Each day we handle:
- 3.5 Million New Posts & Comments From 196 countries
- 39 Million Elasticsearch Queries In 137 languages
- 33 Terabytes of Elasticsearch Data for 175M+ Unique Visitors
- 0.5 billion pageviews
- Millions of events streaming through Kafka, Hive, Impala, eventually Spark
No walls around the garden. Make the Open Web a smarter place.
http://automattic.com/work-with-us/data-wrangler/","created_at":"2015-08-03T19:36:46Z","created_at_i":1438630606,"objectID":"9999197","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"cinquemb"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"I'm working on a tool[0] using LSA [1] to mine the emails from Hacking Team so that people could use it to get more relevant results compared to what wikileaks has available (a lot of stuff to sift through to not know how each message might be related to ones queries outside of just mentioning the word in an email).
Right now I have to break up the term-message matrices by person to do before doing partial eigenvalue decomposition, to generate inverse(sigma) * transpose(u) and inverse(sigma) * transpose(v) and the lower dimensional space representation of each message for each message, but it would be cool to not have to do that if I had more computing power available (a friend let me use his 12 thread/ 6 core machine which has helped a lot while building things).
Ideally it should it hosted somewhere eventually, because the project it self might be a bit complicated/tedious for most people to set up themselves, that would allow people to search it using these indexes as well as enabling (independent) journalists to be able sift through everything in an arguably better way.
[0] https://github.com/cinquemb/hackedteam-email-index-mining
[1] https://en.wikipedia.org/wiki/Latent_semantic_analysis#Deriv..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: $8k on AWS in August for you, any interesting computations?"}},"_tags":["comment","author_cinquemb","story_9997643"],"author":"cinquemb","comment_text":"I'm working on a tool[0] using LSA [1] to mine the emails from Hacking Team so that people could use it to get more relevant results compared to what wikileaks has available (a lot of stuff to sift through to not know how each message might be related to ones queries outside of just mentioning the word in an email).
Right now I have to break up the term-message matrices by person to do before doing partial eigenvalue decomposition, to generate inverse(sigma) * transpose(u) and inverse(sigma) * transpose(v) and the lower dimensional space representation of each message for each message, but it would be cool to not have to do that if I had more computing power available (a friend let me use his 12 thread/ 6 core machine which has helped a lot while building things).
Ideally it should it hosted somewhere eventually, because the project it self might be a bit complicated/tedious for most people to set up themselves, that would allow people to search it using these indexes as well as enabling (independent) journalists to be able sift through everything in an arguably better way.
[0] https://github.com/cinquemb/hackedteam-email-index-mining
[1] https://en.wikipedia.org/wiki/Latent_semantic_analysis#Deriv...","created_at":"2015-08-03T19:20:54Z","created_at_i":1438629654,"objectID":"9999073","parent_id":9997643,"points":null,"story_id":9997643,"story_title":"Ask HN: $8k on AWS in August for you, any interesting computations?","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"socialnature"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"SocialNature | Vancouver, BC, Canada (Remote OK)| Full time | https://www.socialnature.com/
We're looking for Javascript Engineers (Angular.js, Sail.js Node.js)
Join our 5 person team and build the next gen marketing platform for progressive brands. We believe the future of marketing is peer-to-peer, and our platform gives brands the ultimate dashboard into the social web.
We expect you to be driven, equally interested in learning and making an impact. You\u2019ll get an opportunity to work closely with brands, create interesting social integrations, and think about how to store and query big-data.\u00a0We don\u2019t expect you to be an expert in everything already, but we are looking for solid, experienced developers who don\u2019t shy away from new challenges.
If you're interested email me at annalea@socialnature.com"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_socialnature","story_9996333"],"author":"socialnature","comment_text":"SocialNature | Vancouver, BC, Canada (Remote OK)| Full time | https://www.socialnature.com/
We're looking for Javascript Engineers (Angular.js, Sail.js Node.js)
Join our 5 person team and build the next gen marketing platform for progressive brands. We believe the future of marketing is peer-to-peer, and our platform gives brands the ultimate dashboard into the social web.
We expect you to be driven, equally interested in learning and making an impact. You\u2019ll get an opportunity to work closely with brands, create interesting social integrations, and think about how to store and query big-data.\u00a0We don\u2019t expect you to be an expert in everything already, but we are looking for solid, experienced developers who don\u2019t shy away from new challenges.
If you're interested email me at annalea@socialnature.com","created_at":"2015-08-03T19:20:29Z","created_at_i":1438629629,"objectID":"9999069","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tdaltonc"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Imagine this:
A website that fulfills a search query by searching google.com, then searching google.fr, then searching google.cn. It shows the results to the google.com query but gives badges to results based on which governments appear to be censoring them."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Google to France: We Won't Forget It for You Wholesale"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.eff.org/deeplinks/2015/08/google-france-we-wont-forget-it-you-wholesale"}},"_tags":["comment","author_tdaltonc","story_9998818"],"author":"tdaltonc","children":[9999096,10001380],"comment_text":"Imagine this:
A website that fulfills a search query by searching google.com, then searching google.fr, then searching google.cn. It shows the results to the google.com query but gives badges to results based on which governments appear to be censoring them.","created_at":"2015-08-03T19:17:53Z","created_at_i":1438629473,"objectID":"9999051","parent_id":9998818,"points":null,"story_id":9998818,"story_title":"Google to France: We Won't Forget It for You Wholesale","story_url":"https://www.eff.org/deeplinks/2015/08/google-france-we-wont-forget-it-you-wholesale","updated_at":"2023-09-07T03:22:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"bonanza_com"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Bonanza.com is looking for a full-stack developer with the chops to wrangle one of the largest Ruby/Rails apps on the planet. We serve nearly 50,000 merchants, helping them sell 15 million items across numerous buying channels.
The Full Stack Developer at Bonanza will work across a variety of projects, gravitating toward those that are most stimulating for them. Our philosophy is that developers are most productive when they are given projects that best combine their talents & interests.
Because of this customization between "person" and "projects," it is difficult to prescribe in advance the specific tasks that will comprise this job. But here is a representative sample of the challenges we're working on:
* How can we make a buyer's search experience as relevant as possible when 50,000 items match their query?
* How can we build the stickiest possible mobile buyer experience?
* What's it like to use the best toolset for managing an inventory of 100,000 items?
* Can we use natural language recognition to estimate the selling price of an item based on its title and category?
Our challenges range up and down the stack, in a variety of technologies. If you like working at scale, to make improvements in the lives of entrepreneurs, there's plenty for us to get done together.
More info: http://www.bonanza.com/jobs"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_bonanza_com","story_9996333"],"author":"bonanza_com","comment_text":"Bonanza.com is looking for a full-stack developer with the chops to wrangle one of the largest Ruby/Rails apps on the planet. We serve nearly 50,000 merchants, helping them sell 15 million items across numerous buying channels.
The Full Stack Developer at Bonanza will work across a variety of projects, gravitating toward those that are most stimulating for them. Our philosophy is that developers are most productive when they are given projects that best combine their talents & interests.
Because of this customization between "person" and "projects," it is difficult to prescribe in advance the specific tasks that will comprise this job. But here is a representative sample of the challenges we're working on:
* How can we make a buyer's search experience as relevant as possible when 50,000 items match their query?
* How can we build the stickiest possible mobile buyer experience?
* What's it like to use the best toolset for managing an inventory of 100,000 items?
* Can we use natural language recognition to estimate the selling price of an item based on its title and category?
Our challenges range up and down the stack, in a variety of technologies. If you like working at scale, to make improvements in the lives of entrepreneurs, there's plenty for us to get done together.
More info: http://www.bonanza.com/jobs","created_at":"2015-08-03T18:34:27Z","created_at_i":1438626867,"objectID":"9998652","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:21:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tuxracer"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Mac and Windows will 2x all images for users with HiDpi displays. You can use CSS to display the images as background-image then media queries are available to serve differently sized images for HiDPI. Alternatively, most images look completely fine scaled down, they just get blurry scaled up. If you use an image that is 2x bigger than defined in the img tag it will play nice on HiDPI."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Programming fonts"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://s9w.github.io/font_compare/?rp"}},"_tags":["comment","author_tuxracer","story_9996485"],"author":"tuxracer","comment_text":"Mac and Windows will 2x all images for users with HiDpi displays. You can use CSS to display the images as background-image then media queries are available to serve differently sized images for HiDPI. Alternatively, most images look completely fine scaled down, they just get blurry scaled up. If you use an image that is 2x bigger than defined in the img tag it will play nice on HiDPI.","created_at":"2015-08-03T18:32:44Z","created_at_i":1438626764,"objectID":"9998634","parent_id":9998321,"points":null,"story_id":9996485,"story_title":"Programming fonts","story_url":"http://s9w.github.io/font_compare/?rp","updated_at":"2023-09-07T03:21:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"sabarjp"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"The problem is that non-retina images get doubled in size on a retina display, so it looks blurry. I suggest providing 2x sized images. You can use media queries to show different sized images or rely on down-sampling. There's a few different strategies."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Programming fonts"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://s9w.github.io/font_compare/?rp"}},"_tags":["comment","author_sabarjp","story_9996485"],"author":"sabarjp","children":[9998321],"comment_text":"The problem is that non-retina images get doubled in size on a retina display, so it looks blurry. I suggest providing 2x sized images. You can use media queries to show different sized images or rely on down-sampling. There's a few different strategies.","created_at":"2015-08-03T17:55:17Z","created_at_i":1438624517,"objectID":"9998288","parent_id":9998228,"points":null,"story_id":9996485,"story_title":"Programming fonts","story_url":"http://s9w.github.io/font_compare/?rp","updated_at":"2023-09-07T03:21:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gamesbrainiac"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"I feel that this would not deter companies from tracking. People would still use disconnect and adblock to make sure that their search queries are not being tracked. I myself am considering getting a VPN since a lot of sites track according to mac address."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"EFF, AdBlock and Others Launch New \u201cDo Not Track\u201d Standard"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://threatpost.com/eff-adblock-and-others-launch-new-do-not-track-standard/114121"}},"_tags":["comment","author_gamesbrainiac","story_9997533"],"author":"gamesbrainiac","comment_text":"I feel that this would not deter companies from tracking. People would still use disconnect and adblock to make sure that their search queries are not being tracked. I myself am considering getting a VPN since a lot of sites track according to mac address.","created_at":"2015-08-03T17:36:05Z","created_at_i":1438623365,"objectID":"9998075","parent_id":9997533,"points":null,"story_id":9997533,"story_title":"EFF, AdBlock and Others Launch New \u201cDo Not Track\u201d Standard","story_url":"https://threatpost.com/eff-adblock-and-others-launch-new-do-not-track-standard/114121","updated_at":"2023-09-07T03:21:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"lsemel"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Lead Software Engineer
Muck Rack (http://muckrack.com) is looking for a talented individual to join our fast-growing tech startup in New York. Our mission is to make journalists, PR pros and marketers more successful. Muck Rack offers tools that help media professionals find and contact the most relevant journalists for their stories, and monitor how their stories perform.
ABOUT THE POSITION\nYou\u2019ll be working alongside the CTO and engineering team, and with product managers and designers, to execute major technical projects on Muck Rack, lead the building of new features, and help shape our engineering culture and processes.
Our tech stack includes Python, Django, Celery, SASS, Redis, Elasticsearch, Nginx, Gunicorn, Varnish, and Ansible.
This is a full time job based in our headquarters in SoHo, with a flexible work-at-home policy. More info and how to apply: http://sawhorsemedia.com/jobs/lead-software-engineer.html
---
ElasticSearch Expert
We're also looking for an Elasticsearch expert to help with some immediate, high-priority issues as well as long-term projects including improving search queries, improving relevancy, solving glitches we've encountered, and help improve our search interface. Ideally you have deep experience with Elasticsearch, with experience building vertical search engines, solved scaling and relevancy issues, setting up monitoring tools, and can architect and code good search experiences. Email us at jobs@sawhorsemedia.com"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_lsemel","story_9996333"],"author":"lsemel","comment_text":"Lead Software Engineer
Muck Rack (http://muckrack.com) is looking for a talented individual to join our fast-growing tech startup in New York. Our mission is to make journalists, PR pros and marketers more successful. Muck Rack offers tools that help media professionals find and contact the most relevant journalists for their stories, and monitor how their stories perform.
ABOUT THE POSITION\nYou\u2019ll be working alongside the CTO and engineering team, and with product managers and designers, to execute major technical projects on Muck Rack, lead the building of new features, and help shape our engineering culture and processes.
Our tech stack includes Python, Django, Celery, SASS, Redis, Elasticsearch, Nginx, Gunicorn, Varnish, and Ansible.
This is a full time job based in our headquarters in SoHo, with a flexible work-at-home policy. More info and how to apply: http://sawhorsemedia.com/jobs/lead-software-engineer.html
---
ElasticSearch Expert
We're also looking for an Elasticsearch expert to help with some immediate, high-priority issues as well as long-term projects including improving search queries, improving relevancy, solving glitches we've encountered, and help improve our search interface. Ideally you have deep experience with Elasticsearch, with experience building vertical search engines, solved scaling and relevancy issues, setting up monitoring tools, and can architect and code good search experiences. Email us at jobs@sawhorsemedia.com","created_at":"2015-08-03T17:36:02Z","created_at_i":1438623362,"objectID":"9998073","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:21:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Fergi"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"PipelineDB (YC W14) | Jr. Systems Engineer and Sr. Frontend Engineer | San Francisco - ONSITE only | Full time | jobs@pipelinedb.com
We're on a mission to build a new type of database for a modern world in which information is constantly moving, and moving fast. PipelineDB runs SQL queries continuously on large volumes of streaming data, giving companies the capability to easily develop scalable, realtime applications and services using only a familiar SQL interface. No application code is required. This inherently involves solving a lot of big problems, many of which are novel. We\u2019re looking for creative engineers who appreciate the value and freedom of choosing their own projects, approaches, and working with other top talent in a low distraction, streamlined work environment. Our small team has backgrounds from Berkeley, MIT, Facebook, Locu and AdRoll, and we're all doing exactly what we want to be doing: building a groundbreaking new product out of thin air. As an early stage engineer you'll ultimately own a very large part of the product. Which part of the product you take charge of depends on where your interests are, but there are several different potential areas of focus. You'll be entrusted to make sound architectural decisions as well as implement your vision effectively. We are well funded by top investors including SV Angel, Susa Ventures, Data Collective, Paul Buchheit, and more. If you\u2019ve been waiting for an opportunity like this, please send your resume and a quick blurb about yourself to jobs@pipelinedb.com.
Benefits:\n* Full medical/dental/vision insurance * No set work hours--work when you feel smart * Choose your own setup * No vacation policy other than that it is strongly encouraged * Large equity ownership"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_Fergi","story_9996333"],"author":"Fergi","comment_text":"PipelineDB (YC W14) | Jr. Systems Engineer and Sr. Frontend Engineer | San Francisco - ONSITE only | Full time | jobs@pipelinedb.com
We're on a mission to build a new type of database for a modern world in which information is constantly moving, and moving fast. PipelineDB runs SQL queries continuously on large volumes of streaming data, giving companies the capability to easily develop scalable, realtime applications and services using only a familiar SQL interface. No application code is required. This inherently involves solving a lot of big problems, many of which are novel. We\u2019re looking for creative engineers who appreciate the value and freedom of choosing their own projects, approaches, and working with other top talent in a low distraction, streamlined work environment. Our small team has backgrounds from Berkeley, MIT, Facebook, Locu and AdRoll, and we're all doing exactly what we want to be doing: building a groundbreaking new product out of thin air. As an early stage engineer you'll ultimately own a very large part of the product. Which part of the product you take charge of depends on where your interests are, but there are several different potential areas of focus. You'll be entrusted to make sound architectural decisions as well as implement your vision effectively. We are well funded by top investors including SV Angel, Susa Ventures, Data Collective, Paul Buchheit, and more. If you\u2019ve been waiting for an opportunity like this, please send your resume and a quick blurb about yourself to jobs@pipelinedb.com.
Benefits:\n* Full medical/dental/vision insurance * No set work hours--work when you feel smart * Choose your own setup * No vacation policy other than that it is strongly encouraged * Large equity ownership","created_at":"2015-08-03T17:32:42Z","created_at_i":1438623162,"objectID":"9998042","parent_id":9996333,"points":null,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2023-09-07T03:21:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"api"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"I'm using Postgres for a new project. After playing around with various NoSql stores, I've come to the following conclusion:
If you want a super scalable eventually consistent database, use one of the more minimally structured global key value store type DBs like Cassandra. In this case you do not want SQL's guarantees or structure, so don't go there.
If you want a structured database with ACID guarantees, just use XXXXing SQL.
NoSql databases that have attempted to implement structured data, consistency guarantees, rules, ACID or near-ACID semantics, etc., have all started to basically just converge with SQL. They end up re-implementing SQL but with a less consistent, hackier query language and they miss spots. They've gone around the circle and re-invented the wheel and in many cases ended up with an inferior one to boot.
Sure SQL is old. So is math. SQL is rooted in set theory and other pieces of immortal mathematical truth. It's a great example of a software system designed around ageless mathematical concepts that will always be valid. It could use some syntactic modernization, but the core of it will be as useful in a million years as it is today. Learn how to properly structure a database (normalization, DRY, etc.) and how to use its more obscure abilities (esoteric joins) and you'll find that it's amazingly powerful.
PostgreSQL is fantastic because it's doing just that: a bit of modernization around a solid core. It gives you SQL when you want structured data, and it also give you JSON columns when you want to store blobs of unstructured data in the database. So it kind of gives you the best of both worlds: SQL plus a JSON document store. You can (to some extent) query your JSON columns too, though if you intend to do this a lot I'd recommend moving that data into SQL-land.
This facilitates a kind of iterated development where you throw temporary and less structured data into JSON columns, then if you discover later that this data wants to be more long-lived and structured you migrate it to real SQL columns. It's a very agile/YAGNI way of doing things -- do it quick at first, then optimize and clean up once you know what wants to live where and what's really important."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Postgres Guide"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.postgresguide.com/"}},"_tags":["comment","author_api","story_9993237"],"author":"api","children":[9998004],"comment_text":"I'm using Postgres for a new project. After playing around with various NoSql stores, I've come to the following conclusion:
If you want a super scalable eventually consistent database, use one of the more minimally structured global key value store type DBs like Cassandra. In this case you do not want SQL's guarantees or structure, so don't go there.
If you want a structured database with ACID guarantees, just use XXXXing SQL.
NoSql databases that have attempted to implement structured data, consistency guarantees, rules, ACID or near-ACID semantics, etc., have all started to basically just converge with SQL. They end up re-implementing SQL but with a less consistent, hackier query language and they miss spots. They've gone around the circle and re-invented the wheel and in many cases ended up with an inferior one to boot.
Sure SQL is old. So is math. SQL is rooted in set theory and other pieces of immortal mathematical truth. It's a great example of a software system designed around ageless mathematical concepts that will always be valid. It could use some syntactic modernization, but the core of it will be as useful in a million years as it is today. Learn how to properly structure a database (normalization, DRY, etc.) and how to use its more obscure abilities (esoteric joins) and you'll find that it's amazingly powerful.
PostgreSQL is fantastic because it's doing just that: a bit of modernization around a solid core. It gives you SQL when you want structured data, and it also give you JSON columns when you want to store blobs of unstructured data in the database. So it kind of gives you the best of both worlds: SQL plus a JSON document store. You can (to some extent) query your JSON columns too, though if you intend to do this a lot I'd recommend moving that data into SQL-land.
This facilitates a kind of iterated development where you throw temporary and less structured data into JSON columns, then if you discover later that this data wants to be more long-lived and structured you migrate it to real SQL columns. It's a very agile/YAGNI way of doing things -- do it quick at first, then optimize and clean up once you know what wants to live where and what's really important.","created_at":"2015-08-03T17:08:53Z","created_at_i":1438621733,"objectID":"9997812","parent_id":9993237,"story_id":9993237,"story_title":"Postgres Guide","story_url":"http://www.postgresguide.com/","updated_at":"2024-09-19T22:14:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"manndog"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Cognii - Boston, MA
Cognii is building the "Siri for Education" to transform online learning. We are looking for two positions:
1. NLP Engineer
You should have a background or interest in following areas
- natural language processing\n - statistical machine learning\n - syntactic and semantic analysis\n - large scale data modeling\n - information extraction and filtering\n - educational technology experience is a plus\nSome of the responsibilities include:
- Design and implement the scalable web service RESTful API\n - Data pipeline workflow optimization\n - Write different scripts and tools to analyze the usage data logs\n - Experience with Ruby and Rails\nCognii is building the "Siri for Education" to transform online learning. We are looking for two positions:
1. NLP Engineer
You should have a background or interest in following areas
- natural language processing\n - statistical machine learning\n - syntactic and semantic analysis\n - large scale data modeling\n - information extraction and filtering\n - educational technology experience is a plus\nSome of the responsibilities include:
- Design and implement the scalable web service RESTful API\n - Data pipeline workflow optimization\n - Write different scripts and tools to analyze the usage data logs\n - Experience with Ruby and Rails\n----- About Keen IO -----
Keen IO is an API platform that lets developers collect and study custom events at a massive scale.
Our Mission = The Discovery of Truth.
Some things we believe in:\n\u2022 Honesty & empathy, introspection, Distributed innovation, the importance of laughter & relationships, play to your strengths; patch your weaknesses with diverse collaborators.
----- Platform & Middleware Engineers -----
(SF Bay Area or Remote)\nKeen IO is seeking engineers to join our team. We build and nurture an Analytics API that processes user data and queries around the clock, across diverse industries. Whether it\u2019s gaming, e-commerce, advertising, publishing, or IoT, our platform aims to make Analytics so easy for developers that they can focus entirely on their product. Our challenge is to scale with a constantly growing data deluge and evolve along with our user\u2019s needs.
Skills: Cassandra, Scala, Python, Java, Tornado, Go, Zookeeper, Storm, Kafka, Mesos
----- Consultative Sales -----
(SF Bay Area Only)\nBe one of the first sales reps for a fast-growing, Sequoia-backed, cloud analytics platform.
Our Cross-Vertical Practice is a small, cross-functional team (marketing, sales, sales engineering, data science) tasked with growing our business across all verticals.
For questions, please contact us at humans@keen.io or visit us on Angel List (https://angel.co/keen-io/jobs)."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_truongor17","story_9996333"],"author":"truongor17","children":[9997640],"comment_text":"SF Bay Area or Remote || Keen IO || https://keen.io/
----- About Keen IO -----
Keen IO is an API platform that lets developers collect and study custom events at a massive scale.
Our Mission = The Discovery of Truth.
Some things we believe in:\n\u2022 Honesty & empathy, introspection, Distributed innovation, the importance of laughter & relationships, play to your strengths; patch your weaknesses with diverse collaborators.
----- Platform & Middleware Engineers -----
(SF Bay Area or Remote)\nKeen IO is seeking engineers to join our team. We build and nurture an Analytics API that processes user data and queries around the clock, across diverse industries. Whether it\u2019s gaming, e-commerce, advertising, publishing, or IoT, our platform aims to make Analytics so easy for developers that they can focus entirely on their product. Our challenge is to scale with a constantly growing data deluge and evolve along with our user\u2019s needs.
Skills: Cassandra, Scala, Python, Java, Tornado, Go, Zookeeper, Storm, Kafka, Mesos
----- Consultative Sales -----
(SF Bay Area Only)\nBe one of the first sales reps for a fast-growing, Sequoia-backed, cloud analytics platform.
Our Cross-Vertical Practice is a small, cross-functional team (marketing, sales, sales engineering, data science) tasked with growing our business across all verticals.
For questions, please contact us at humans@keen.io or visit us on Angel List (https://angel.co/keen-io/jobs).","created_at":"2015-08-03T16:13:30Z","created_at_i":1438618410,"objectID":"9997197","parent_id":9996333,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2024-09-19T22:14:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dahart"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"You're probably right, but nobody I know calls it "Whitted ray tracing" unless Turner is there in the room, or some point is being made about or in reference to his paper specifically. ;) In my experience, graphics people generally do mean producing colors for pixels when they say "ray tracing". But I think physicists would assume "ray tracing" means visibility queries and nothing more.
When someone says 'ray tracing' to me today, before they elaborate on what they mean, I assume they're talking about using straight line segment visibility tests probably to make some kind of picture, probably using Monte Carlo techniques, probably for global illumination. So, they might mean path tracing, or one of the (many) other ways to simulate lighting, or my assumption might be wrong, but that's okay because language is fun, and vague, overridden and imperfect terms allow us to have longer conversations. :)"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Disney's Hyperion Renderer"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.disneyanimation.com/technology/innovations/hyperion"}},"_tags":["comment","author_dahart","story_9994032"],"author":"dahart","comment_text":"You're probably right, but nobody I know calls it "Whitted ray tracing" unless Turner is there in the room, or some point is being made about or in reference to his paper specifically. ;) In my experience, graphics people generally do mean producing colors for pixels when they say "ray tracing". But I think physicists would assume "ray tracing" means visibility queries and nothing more.
When someone says 'ray tracing' to me today, before they elaborate on what they mean, I assume they're talking about using straight line segment visibility tests probably to make some kind of picture, probably using Monte Carlo techniques, probably for global illumination. So, they might mean path tracing, or one of the (many) other ways to simulate lighting, or my assumption might be wrong, but that's okay because language is fun, and vague, overridden and imperfect terms allow us to have longer conversations. :)","created_at":"2015-08-03T16:00:42Z","created_at_i":1438617642,"objectID":"9997032","parent_id":9995843,"story_id":9994032,"story_title":"Disney's Hyperion Renderer","story_url":"http://www.disneyanimation.com/technology/innovations/hyperion","updated_at":"2024-09-19T22:14:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mopatches"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Periscope -- Customer Support Specialist | San Francisco | ONSITE | FULLTIME
Periscope is a data analysis tool for professional data analysts that allows them to type SQL queries and get charts really, really fast.
Analysts spend their lives in Periscope, doing ad-hoc analyses, putting together dashboards to share to external clients, and displaying KPIs to entire companies.
Your job is to help them do that.
https://www.periscope.io/customer-support-specialist"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (August 2015)"}},"_tags":["comment","author_mopatches","story_9996333"],"author":"mopatches","comment_text":"Periscope -- Customer Support Specialist | San Francisco | ONSITE | FULLTIME
Periscope is a data analysis tool for professional data analysts that allows them to type SQL queries and get charts really, really fast.
Analysts spend their lives in Periscope, doing ad-hoc analyses, putting together dashboards to share to external clients, and displaying KPIs to entire companies.
Your job is to help them do that.
https://www.periscope.io/customer-support-specialist","created_at":"2015-08-03T15:32:34Z","created_at_i":1438615954,"objectID":"9996712","parent_id":9996333,"story_id":9996333,"story_title":"Ask HN: Who is hiring? (August 2015)","updated_at":"2024-09-19T22:14:33Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"sbuttgereit"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Yes, this is exactly why I think a guide examining such a topic would be good. You get a doc page for it (see icebraining's comment), but that's about it. Performance gets covered often, thinking about how to get the most of PostgreSQL's type system doesn't.
I don't really use this feature to drive structure (really at all), but do use it extensively when programming functions in PL/pgSQL. I also exploit this in queries sometimes as well. It's not a life changing feature to be sure, but it is relatively unique and can help solve problems that are more cumbersome to solve without the feature."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Postgres Guide"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.postgresguide.com/"}},"_tags":["comment","author_sbuttgereit","story_9993237"],"author":"sbuttgereit","comment_text":"Yes, this is exactly why I think a guide examining such a topic would be good. You get a doc page for it (see icebraining's comment), but that's about it. Performance gets covered often, thinking about how to get the most of PostgreSQL's type system doesn't.
I don't really use this feature to drive structure (really at all), but do use it extensively when programming functions in PL/pgSQL. I also exploit this in queries sometimes as well. It's not a life changing feature to be sure, but it is relatively unique and can help solve problems that are more cumbersome to solve without the feature.","created_at":"2015-08-03T13:32:20Z","created_at_i":1438608740,"objectID":"9995610","parent_id":9994247,"story_id":9993237,"story_title":"Postgres Guide","story_url":"http://www.postgresguide.com/","updated_at":"2024-09-19T22:14:28Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"msandford"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"I'm not sure that repealing Glass-Steagall was the problem. We did have the savings & loan crisis and that was prior to the repeal.
Personally I think it has more to do with the banks going public and then being on the treadmill of ever increasing profits. The NYT has a piece on the issue from 1999.
http://query.nytimes.com/gst/fullpage.html?res=9C03E7DF1639F...
Slate also documents the banks going public, again prior to repealing Glass-Steagall.
http://www.slate.com/articles/business/moneybox/2010/01/the_..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Defending the One Percent"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.27.3.21"}},"_tags":["comment","author_msandford","story_9994967"],"author":"msandford","children":[10011833],"comment_text":"I'm not sure that repealing Glass-Steagall was the problem. We did have the savings & loan crisis and that was prior to the repeal.
Personally I think it has more to do with the banks going public and then being on the treadmill of ever increasing profits. The NYT has a piece on the issue from 1999.
http://query.nytimes.com/gst/fullpage.html?res=9C03E7DF1639F...
Slate also documents the banks going public, again prior to repealing Glass-Steagall.
http://www.slate.com/articles/business/moneybox/2010/01/the_...","created_at":"2015-08-03T13:30:37Z","created_at_i":1438608637,"objectID":"9995601","parent_id":9995430,"story_id":9994967,"story_title":"Defending the One Percent","story_url":"http://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.27.3.21","updated_at":"2024-09-19T22:14:28Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"angdis"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Yes, I've been a mint user since well before they were bought by Intuit.
I didn't think it was a bad idea, because it is clearly explained that Mint can't make changes to your accounts. It is only used for query. I am under the impression that the banks only allow Mint query capabilities. They've lately released a new service called "Mint Bills"-- which does make changes but that is separate from their main "Mint" service.
I've recommended Mint to friends and those who've tried it like it.
Things are changing lately, however. Banks are providing more and more "Mint-like" analytic services for their customers. These days, if you have your stuff at one bank there's little need for something like Mint.
I think there will always be room for a service that can work with multiple financial services/banks/accounts at the same time and with a uniform interface. Unfortunately, this is an exceedingly hard business for start-ups to crack (my opinion). To do it right (without asking users to literally surrender control of their accounts), services like Mint need to negotiate with multiple financial institutions-- not fun at all, just to get to the starting-point where one can compete with Mint."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Why would people trust their banking credentials to a service like Mint?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":""}},"_tags":["comment","author_angdis","story_9995304"],"author":"angdis","children":[9995956,9996448],"comment_text":"Yes, I've been a mint user since well before they were bought by Intuit.
I didn't think it was a bad idea, because it is clearly explained that Mint can't make changes to your accounts. It is only used for query. I am under the impression that the banks only allow Mint query capabilities. They've lately released a new service called "Mint Bills"-- which does make changes but that is separate from their main "Mint" service.
I've recommended Mint to friends and those who've tried it like it.
Things are changing lately, however. Banks are providing more and more "Mint-like" analytic services for their customers. These days, if you have your stuff at one bank there's little need for something like Mint.
I think there will always be room for a service that can work with multiple financial services/banks/accounts at the same time and with a uniform interface. Unfortunately, this is an exceedingly hard business for start-ups to crack (my opinion). To do it right (without asking users to literally surrender control of their accounts), services like Mint need to negotiate with multiple financial institutions-- not fun at all, just to get to the starting-point where one can compete with Mint.","created_at":"2015-08-03T13:18:33Z","created_at_i":1438607913,"objectID":"9995517","parent_id":9995304,"story_id":9995304,"story_title":"Ask HN: Why would people trust their banking credentials to a service like Mint?","story_url":"","updated_at":"2024-09-19T22:14:28Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"onion2k"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Except in the case of Facebook, they don't go further afield. People stay within Facebook's walled garden, posting updates in Facebook, liking brands in Facebook, talking to their friends in Facebook, playing games in Facebook ... they don't leave.
To me this raises an interesting philosophical question - is Facebook actually separate to the internet? It uses the underlying transport mechanisms of the net, sure, but that's not really what we mean when we say the internet. When my mother says she's using the internet she doesn't mean she's generating HTTP GET requests to query servers and render the resulting HTML. She means she's using a browser to read a page and then clicking on a link to get another page. Facebook certainly isn't the world wide web or email, which is what most people mean when they say "internet". If what you're doing is in an largely separate, walled-off area, that you mostly access with an app rather than a browser, is that really what we mean when we say the internet?
I'd need to think about it more, but I imagine there's quite a compelling argument that using Facebook is not using the internet for any non-technical definition of 'the internet'."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Millions of Facebook users have no idea they\u2019re using the internet"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://qz.com/333313/milliions-of-facebook-users-have-no-idea-theyre-using-the-internet/"}},"_tags":["comment","author_onion2k","story_9995043"],"author":"onion2k","children":[9995248,9995342,9996144],"comment_text":"Except in the case of Facebook, they don't go further afield. People stay within Facebook's walled garden, posting updates in Facebook, liking brands in Facebook, talking to their friends in Facebook, playing games in Facebook ... they don't leave.
To me this raises an interesting philosophical question - is Facebook actually separate to the internet? It uses the underlying transport mechanisms of the net, sure, but that's not really what we mean when we say the internet. When my mother says she's using the internet she doesn't mean she's generating HTTP GET requests to query servers and render the resulting HTML. She means she's using a browser to read a page and then clicking on a link to get another page. Facebook certainly isn't the world wide web or email, which is what most people mean when they say "internet". If what you're doing is in an largely separate, walled-off area, that you mostly access with an app rather than a browser, is that really what we mean when we say the internet?
I'd need to think about it more, but I imagine there's quite a compelling argument that using Facebook is not using the internet for any non-technical definition of 'the internet'.","created_at":"2015-08-03T12:24:50Z","created_at_i":1438604690,"objectID":"9995216","parent_id":9995125,"story_id":9995043,"story_title":"Millions of Facebook users have no idea they\u2019re using the internet","story_url":"http://qz.com/333313/milliions-of-facebook-users-have-no-idea-theyre-using-the-internet/","updated_at":"2024-09-19T22:14:28Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Rambunctious"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Well, one threat for Google's search business is possibly Apple. By controlling the device, browser and the platform - they have an option to integrate their own search offering instead of relying on third parties. Not sure what the Topsy acquisition was about - but perhaps there is some sort of a social search integration in the works.
And then there's Facebook. The key for them is to figure out a viable business out of head search queries i.e. instead of going after all possible searches. Perhaps mainly target shopping oriented search queries via recommendations from friends?
Of course, this is all easier said than done. Apple has seen its struggles with Maps and even Google couldn't do much with Google+."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Do you think there will be a full replacement to Google?"}},"_tags":["comment","author_Rambunctious","story_9994253"],"author":"Rambunctious","comment_text":"Well, one threat for Google's search business is possibly Apple. By controlling the device, browser and the platform - they have an option to integrate their own search offering instead of relying on third parties. Not sure what the Topsy acquisition was about - but perhaps there is some sort of a social search integration in the works.
And then there's Facebook. The key for them is to figure out a viable business out of head search queries i.e. instead of going after all possible searches. Perhaps mainly target shopping oriented search queries via recommendations from friends?
Of course, this is all easier said than done. Apple has seen its struggles with Maps and even Google couldn't do much with Google+.","created_at":"2015-08-03T09:44:21Z","created_at_i":1438595061,"objectID":"9994700","parent_id":9994253,"story_id":9994253,"story_title":"Ask HN: Do you think there will be a full replacement to Google?","updated_at":"2024-09-19T22:14:22Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jagira"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"I built a kind of similar stuff as a side project in 2011 - [redacted]. It allowed you to leave notes for a URL which your friends / followers can see when they visit that URL.
In the beginning when I was testing this with my friends and colleagues, I sent every URL a user visited to the server to check if any of his friends have left any notes and then alert him via notification badges. I disabled it when I started seeing a lot of private URLs (like Google Docs links with share access) in server logs. I then changed the extension to query server only when a user clicks on extension button.
This made it a bit safer, but the extension still needed access to all the sites a user visits. And with Chrome's auto updation of extensions, one may never know if the extension author has started sending every URL back to server again.
After developing such extension, I am quite suspicious such extensions and only install extensions from trusted authors (Buffer, Pocket, etc)."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.metafruit.com/kiwi"}},"_tags":["comment","author_jagira","story_9992735"],"author":"jagira","children":[9994175],"comment_text":"I built a kind of similar stuff as a side project in 2011 - [redacted]. It allowed you to leave notes for a URL which your friends / followers can see when they visit that URL.
In the beginning when I was testing this with my friends and colleagues, I sent every URL a user visited to the server to check if any of his friends have left any notes and then alert him via notification badges. I disabled it when I started seeing a lot of private URLs (like Google Docs links with share access) in server logs. I then changed the extension to query server only when a user clicks on extension button.
This made it a bit safer, but the extension still needed access to all the sites a user visits. And with Chrome's auto updation of extensions, one may never know if the extension author has started sending every URL back to server again.
After developing such extension, I am quite suspicious such extensions and only install extensions from trusted authors (Buffer, Pocket, etc).","created_at":"2015-08-03T06:40:19Z","created_at_i":1438584019,"objectID":"9994150","parent_id":9992735,"story_id":9992735,"story_title":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf","story_url":"http://www.metafruit.com/kiwi","updated_at":"2024-09-19T22:14:22Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"atotic"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"I might enjoy using this. But, PRIVACY! Sending back every visited URL has never been ok for any reason, first time I saw this idea shot down was in '93.
But there might be a way out:
I'd be willing to give up privacy of URL hashes. This is how I'd do it:
- you already track a set of URLs that have discussions (I assume). If not, you need to figure out how to seed these. Volunteers, APIs....
- hash these URLs on server, and use a not-too-unique hash function. You want to end up with a high collision rate, but not too high.
- now, the client can query for conversations without revealing the URL it has visited:\n - ask server whether there are any conversations for a particular hash.\n - if server finds any, it returns { pageUrl: '', conversationsUrls[]} \n - now client can decide whether the url really matches, or it was just a random hash collision.
- I know this is not perfect. A privacy-busting determined enemy could generate hashes of large number of public sites and use statistics to infer what sites you've visited just from your hashes. But it'd be good enough for me.
Bonus money-making idea:\n- offer your plugin as a paid service to different web communities. Increases their "community engagement".
I seriously contemplated starting an "annotations" startup in the 90s. Someone else did, and they folded after a few years."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.metafruit.com/kiwi"}},"_tags":["comment","author_atotic","story_9992735"],"author":"atotic","children":[9994136],"comment_text":"I might enjoy using this. But, PRIVACY! Sending back every visited URL has never been ok for any reason, first time I saw this idea shot down was in '93.
But there might be a way out:
I'd be willing to give up privacy of URL hashes. This is how I'd do it:
- you already track a set of URLs that have discussions (I assume). If not, you need to figure out how to seed these. Volunteers, APIs....
- hash these URLs on server, and use a not-too-unique hash function. You want to end up with a high collision rate, but not too high.
- now, the client can query for conversations without revealing the URL it has visited:\n - ask server whether there are any conversations for a particular hash.\n - if server finds any, it returns { pageUrl: '', conversationsUrls[]} \n - now client can decide whether the url really matches, or it was just a random hash collision.
- I know this is not perfect. A privacy-busting determined enemy could generate hashes of large number of public sites and use statistics to infer what sites you've visited just from your hashes. But it'd be good enough for me.
Bonus money-making idea:\n- offer your plugin as a paid service to different web communities. Increases their "community engagement".
I seriously contemplated starting an "annotations" startup in the 90s. Someone else did, and they folded after a few years.","created_at":"2015-08-03T04:55:38Z","created_at_i":1438577738,"objectID":"9993892","parent_id":9993229,"points":null,"story_id":9992735,"story_title":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf","story_url":"http://www.metafruit.com/kiwi","updated_at":"2023-09-07T03:21:35Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"pault"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"They don't wrap when their parent container shrinks, but sometimes that's the behavior you want (navigation bars for example), and media queries get around those issues nicely."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"\u201cGoogles 'vertical center div css' for the 100th time. reads for 7 minutes\u201d"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.bleonard.com/blog/2015/08/01/teaching-css/"}},"_tags":["comment","author_pault","story_9993171"],"author":"pault","comment_text":"They don't wrap when their parent container shrinks, but sometimes that's the behavior you want (navigation bars for example), and media queries get around those issues nicely.","created_at":"2015-08-03T02:34:52Z","created_at_i":1438569292,"objectID":"9993423","parent_id":9993389,"points":null,"story_id":9993171,"story_title":"\u201cGoogles 'vertical center div css' for the 100th time. reads for 7 minutes\u201d","story_url":"http://www.bleonard.com/blog/2015/08/01/teaching-css/","updated_at":"2023-09-07T03:21:35Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"thinkcontext"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"In my extension (thinkcontext) to get around leaking browsing behavior to 3rd party sites I download the complete data set. That way all queries are local which has the added benefit of low latency."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.metafruit.com/kiwi"}},"_tags":["comment","author_thinkcontext","story_9992735"],"author":"thinkcontext","children":[9993328],"comment_text":"In my extension (thinkcontext) to get around leaking browsing behavior to 3rd party sites I download the complete data set. That way all queries are local which has the added benefit of low latency.","created_at":"2015-08-03T01:41:30Z","created_at_i":1438566090,"objectID":"9993259","parent_id":9992866,"points":null,"story_id":9992735,"story_title":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf","story_url":"http://www.metafruit.com/kiwi","updated_at":"2023-09-07T03:21:35Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"curiousjorge"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"does the searching happen on your machine (scrape google search results by crafting a url query) or does it get routed to a central server that we are forced to trust? If the latter, no way in hell this is going to be popular around here."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.metafruit.com/kiwi"}},"_tags":["comment","author_curiousjorge","story_9992735"],"author":"curiousjorge","children":[9992973],"comment_text":"does the searching happen on your machine (scrape google search results by crafting a url query) or does it get routed to a central server that we are forced to trust? If the latter, no way in hell this is going to be popular around here.","created_at":"2015-08-02T23:35:34Z","created_at_i":1438558534,"objectID":"9992951","parent_id":9992735,"points":null,"story_id":9992735,"story_title":"Show HN: Chrome/FF plugin brings the HN and Reddit conversation to you as you surf","story_url":"http://www.metafruit.com/kiwi","updated_at":"2023-09-07T03:21:30Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jasonlotito"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"> Yes, Apple has it's issues.
What's interesting is the ease of which I can replace Microsoft with Apple here.
"From the moment an account is created, Apple begins watching. The company saves customers\u2019 basic information\u2014name, contact details, passwords, demographic data and credit card specifics \u2014but it also digs a bit deeper.
Other information Apple saves includes search queries and conversations with the new digital personal assistant Siri; contents of private communications such as email; websites and apps visited (including features accessed and length of time used); and contents of private folders."
The article makes it clear this isn't just Microsoft:
"\u201cBoth Siri and Google Now require access to the user\u2019s personal information to personalise responses, while both Apple and Google offer developers the ability to deliver personalised ads to users based on information such as app installs.\u201d
I guess what this means is if you are earnest in your concern for privacy and security, neither Apple nor Microsoft are in any way possible solutions for you."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Using Windows 10? Microsoft is watching"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.newsweek.com/windows-10-recording-users-every-move-358952"}},"_tags":["comment","author_jasonlotito","story_9991785"],"author":"jasonlotito","comment_text":"> Yes, Apple has it's issues.
What's interesting is the ease of which I can replace Microsoft with Apple here.
"From the moment an account is created, Apple begins watching. The company saves customers\u2019 basic information\u2014name, contact details, passwords, demographic data and credit card specifics \u2014but it also digs a bit deeper.
Other information Apple saves includes search queries and conversations with the new digital personal assistant Siri; contents of private communications such as email; websites and apps visited (including features accessed and length of time used); and contents of private folders."
The article makes it clear this isn't just Microsoft:
"\u201cBoth Siri and Google Now require access to the user\u2019s personal information to personalise responses, while both Apple and Google offer developers the ability to deliver personalised ads to users based on information such as app installs.\u201d
I guess what this means is if you are earnest in your concern for privacy and security, neither Apple nor Microsoft are in any way possible solutions for you.","created_at":"2015-08-02T20:34:20Z","created_at_i":1438547660,"objectID":"9992352","parent_id":9992251,"points":null,"story_id":9991785,"story_title":"Using Windows 10? Microsoft is watching","story_url":"http://www.newsweek.com/windows-10-recording-users-every-move-358952","updated_at":"2023-09-07T03:21:30Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Twirrim"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"maybe I should rephrase that as one outbound DNS query? The answer should be cached locally, or at least at your next hop resolver, and the TTL of 300 seconds is long enough to make it effective"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Why and how is Hacker News so fast?"}},"_tags":["comment","author_Twirrim","story_9990630"],"author":"Twirrim","comment_text":"maybe I should rephrase that as one outbound DNS query? The answer should be cached locally, or at least at your next hop resolver, and the TTL of 300 seconds is long enough to make it effective","created_at":"2015-08-02T15:51:10Z","created_at_i":1438530670,"objectID":"9991101","parent_id":9991023,"points":null,"story_id":9990630,"story_title":"Ask HN: Why and how is Hacker News so fast?","updated_at":"2023-09-07T03:21:25Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ColinWright"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"A couple of months ago I posted this[0]:
The Point of the Banach-Tarski Theorem\n \u2013 not just a curiosity\nThe Banach-Tarski theorem is a lovely result, and I look forward to seeing what people say about this new presentation of it.
========
[0] https://news.ycombinator.com/item?id=9674286
[1] https://hn.algolia.com/?query=banach%20tarski&sort=byDate&pr..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"The Banach\u2013Tarski Paradox [video]"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.youtube.com/watch?v=s86-Z-CbaHA"}},"_tags":["comment","author_ColinWright","story_9990992"],"author":"ColinWright","comment_text":"A couple of months ago I posted this[0]:
The Point of the Banach-Tarski Theorem\n \u2013 not just a curiosity\nThe Banach-Tarski theorem is a lovely result, and I look forward to seeing what people say about this new presentation of it.
========
[0] https://news.ycombinator.com/item?id=9674286
[1] https://hn.algolia.com/?query=banach%20tarski&sort=byDate&pr...","created_at":"2015-08-02T15:48:56Z","created_at_i":1438530536,"objectID":"9991090","parent_id":9990992,"points":null,"story_id":9990992,"story_title":"The Banach\u2013Tarski Paradox [video]","story_url":"https://www.youtube.com/watch?v=s86-Z-CbaHA","updated_at":"2023-09-07T03:21:25Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"fapjacks"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"On the other hand, they do some stupid stuff, disregarding the standard and how useful it is [0].
[0] https://blog.cloudflare.com/deprecating-dns-any-meta-query-t..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Why and how is Hacker News so fast?"}},"_tags":["comment","author_fapjacks","story_9990630"],"author":"fapjacks","children":[9991174],"comment_text":"On the other hand, they do some stupid stuff, disregarding the standard and how useful it is [0].
[0] https://blog.cloudflare.com/deprecating-dns-any-meta-query-t...","created_at":"2015-08-02T15:27:01Z","created_at_i":1438529221,"objectID":"9991034","parent_id":9990840,"points":null,"story_id":9990630,"story_title":"Ask HN: Why and how is Hacker News so fast?","updated_at":"2023-09-07T03:21:25Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ised"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":""... (so just one DNS request)."
Are you sure?
request #1
1 news.ycombinator.com:\n 207 bytes, 1+0+4+2 records, response, noerror\n query: 1 news.ycombinator.com\n authority: ycombinator.com 172800 NS ns-225.awsdns-28.com
request #2
1 news.ycombinator.com:\n 225 bytes, 1+1+4+0 records, response, authoritative, noerror\n query: 1 news.ycombinator.com\n answer: news.ycombinator.com 300 CNAME news.ycombinator.com.cdn.cloudflare.net\nauthority: ycombinator.com 172800 NS ns-1411.awsdns-48.org
request #3
1 news.ycombinator.com.cdn.cloudflare.net:\n 367 bytes, 1+0+5+10 records, response, noerror\n query: 1 news.ycombinator.com.cdn.cloudflare.net\n authority: cloudflare.net 172800 NS ns1.cloudflare.net
request #4
1 news.ycombinator.com.cdn.cloudflare.net:\n 89 bytes, 1+2+0+0 records, response, authoritative, noerror\n query: 1 news.ycombinator.com.cdn.cloudflare.net\n answer: news.ycombinator.com.cdn.cloudflare.net 300 A 198.41.190.47
A HOSTS file entry for HN will avoid this routine."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Why and how is Hacker News so fast?"}},"_tags":["comment","author_ised","story_9990630"],"author":"ised","children":[9991101],"comment_text":""... (so just one DNS request)."
Are you sure?
request #1
1 news.ycombinator.com:\n 207 bytes, 1+0+4+2 records, response, noerror\n query: 1 news.ycombinator.com\n authority: ycombinator.com 172800 NS ns-225.awsdns-28.com
request #2
1 news.ycombinator.com:\n 225 bytes, 1+1+4+0 records, response, authoritative, noerror\n query: 1 news.ycombinator.com\n answer: news.ycombinator.com 300 CNAME news.ycombinator.com.cdn.cloudflare.net\nauthority: ycombinator.com 172800 NS ns-1411.awsdns-48.org
request #3
1 news.ycombinator.com.cdn.cloudflare.net:\n 367 bytes, 1+0+5+10 records, response, noerror\n query: 1 news.ycombinator.com.cdn.cloudflare.net\n authority: cloudflare.net 172800 NS ns1.cloudflare.net
request #4
1 news.ycombinator.com.cdn.cloudflare.net:\n 89 bytes, 1+2+0+0 records, response, authoritative, noerror\n query: 1 news.ycombinator.com.cdn.cloudflare.net\n answer: news.ycombinator.com.cdn.cloudflare.net 300 A 198.41.190.47
A HOSTS file entry for HN will avoid this routine.","created_at":"2015-08-02T15:23:06Z","created_at_i":1438528986,"objectID":"9991023","parent_id":9990893,"points":null,"story_id":9990630,"story_title":"Ask HN: Why and how is Hacker News so fast?","updated_at":"2023-09-07T03:21:25Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mattjaynes"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Wow, this is awesome. Unfortunately it will probably be a good wait before this is built and released to the public.
In the meantime, I've been developing a framework using EC2, S3, S3DFS (DFS = Distributed File System) along with SQLite to try and solve the database problem on AWS.
S3DFS allows you to mount an S3 bucket to one or multiple EC2 instances as if it were a local filesystem. It works at the block level and has read/write caching - so is very fast. (Note: It's free to use for development, but otherwise requires a paid commercial license.)
Since SQLite uses flat files and has no database server or client, it is ideal for use on S3. SQLite only scales well to a point, so to get around its limitations I am separating each user into their own database (each database is a separate flat file). This can sometimes be a bad idea, but SQLite allows you to attach multiple databases together to essentially create a temporary master database where you can run queries across all the user databases at once. This will be handy for search indexing, site-wide stats, etc. In order to change the user databases I will have utilities to allow global scheme alterations.
So, by adding EC2 instances that are all attached to the same S3 bucket, and using round-robin dns to distribute traffic among them, you can theoretically scale a web service infinitely. Just launch a new EC2 instance and add it to the round-robin dns pool and you're done! Even that could be automated to respond dynamically to traffic.
Imagine the long-term cost-savings from only having to build your app and architecture once and not having to go through a more complex scaling process like this: http://www.scribd.com/doc/21793/Scaling-PHPMySQL-Presentation-from-Flickr
Certainly you should only worry about scaling after your app is up and running, but what if you didn't have to ever worry about it? That's the goal of this framework.
This is pre-alpha of course and I'm still in the development and experimentation stage, but I wanted to share with the community and get some feedback!
http://www.openfount.com/blog/s3dfs-for-ec2
http://docs.amazonwebservices.com/AmazonEC2/gsg/2006-06-26/
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/gsg/
Update: Since this was so lengthy, I also added it as a post here: http://blog.nanobeepers.com/2007/04/07/infinitely-scalable-framework-with-aws/
Update2: Please leave any feedback here: http://news.ycombinator.com/comments?id=10001 "},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Amazon reveals new web service: Amazon Simple Database"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://seattle.craigslist.org/see/eng/299279879.html"}},"_tags":["comment","author_mattjaynes","story_9969"],"author":"mattjaynes","comment_text":"Wow, this is awesome. Unfortunately it will probably be a good wait before this is built and released to the public.
In the meantime, I've been developing a framework using EC2, S3, S3DFS (DFS = Distributed File System) along with SQLite to try and solve the database problem on AWS.
S3DFS allows you to mount an S3 bucket to one or multiple EC2 instances as if it were a local filesystem. It works at the block level and has read/write caching - so is very fast. (Note: It's free to use for development, but otherwise requires a paid commercial license.)
Since SQLite uses flat files and has no database server or client, it is ideal for use on S3. SQLite only scales well to a point, so to get around its limitations I am separating each user into their own database (each database is a separate flat file). This can sometimes be a bad idea, but SQLite allows you to attach multiple databases together to essentially create a temporary master database where you can run queries across all the user databases at once. This will be handy for search indexing, site-wide stats, etc. In order to change the user databases I will have utilities to allow global scheme alterations.
So, by adding EC2 instances that are all attached to the same S3 bucket, and using round-robin dns to distribute traffic among them, you can theoretically scale a web service infinitely. Just launch a new EC2 instance and add it to the round-robin dns pool and you're done! Even that could be automated to respond dynamically to traffic.
Imagine the long-term cost-savings from only having to build your app and architecture once and not having to go through a more complex scaling process like this: http://www.scribd.com/doc/21793/Scaling-PHPMySQL-Presentation-from-Flickr
Certainly you should only worry about scaling after your app is up and running, but what if you didn't have to ever worry about it? That's the goal of this framework.
This is pre-alpha of course and I'm still in the development and experimentation stage, but I wanted to share with the community and get some feedback!
http://www.openfount.com/blog/s3dfs-for-ec2
http://docs.amazonwebservices.com/AmazonEC2/gsg/2006-06-26/
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/gsg/
Update: Since this was so lengthy, I also added it as a post here: http://blog.nanobeepers.com/2007/04/07/infinitely-scalable-framework-with-aws/
Update2: Please leave any feedback here: http://news.ycombinator.com/comments?id=10001 ","created_at":"2007-04-07T10:40:43Z","created_at_i":1175942443,"objectID":"9991","parent_id":9969,"story_id":9969,"story_title":"Amazon reveals new web service: Amazon Simple Database","story_url":"http://seattle.craigslist.org/see/eng/299279879.html","updated_at":"2024-09-19T16:18:45Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Twirrim"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"It's relatively straightforward. Take a look at what Pingdom tells you it does when the site loads:
http://tools.pingdom.com/fpt/#!/cnYAuq/https://news.ycombina...
5 requests, Total size 12.6kB, total time 661ms. 1 request for the content, 1 for css, and then 3 gifs, all from a single domain (so just one DNS request). If you look at the profile, 47% of the time was spent doing that one DNS lookup, so subsequent requests will be even quicker (plus caching will handle those gifs just fine)
That's almost no work for the end client to do. No javsacript for the client to download, parse and process, just straight content to render and done.
The reduced request count is especially important when it comes to the mobile experience, where bandwidth might be great, but latency is terrible. Every request you have to make to render a page significantly impacts the loading time (HTTP2 helps here). Along with trying to reduce the number of resources being loaded on the screen, it's especially important to reduce the number of domains you're getting them from so you keep the DNS queries down."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Why and how is Hacker News so fast?"}},"_tags":["comment","author_Twirrim","story_9990630"],"author":"Twirrim","children":[9990984,9991023,9994518],"comment_text":"It's relatively straightforward. Take a look at what Pingdom tells you it does when the site loads:
http://tools.pingdom.com/fpt/#!/cnYAuq/https://news.ycombina...
5 requests, Total size 12.6kB, total time 661ms. 1 request for the content, 1 for css, and then 3 gifs, all from a single domain (so just one DNS request). If you look at the profile, 47% of the time was spent doing that one DNS lookup, so subsequent requests will be even quicker (plus caching will handle those gifs just fine)
That's almost no work for the end client to do. No javsacript for the client to download, parse and process, just straight content to render and done.
The reduced request count is especially important when it comes to the mobile experience, where bandwidth might be great, but latency is terrible. Every request you have to make to render a page significantly impacts the loading time (HTTP2 helps here). Along with trying to reduce the number of resources being loaded on the screen, it's especially important to reduce the number of domains you're getting them from so you keep the DNS queries down.","created_at":"2015-08-02T14:48:08Z","created_at_i":1438526888,"objectID":"9990893","parent_id":9990630,"story_id":9990630,"story_title":"Ask HN: Why and how is Hacker News so fast?","updated_at":"2024-09-19T22:13:51Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Hello71"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"although it's useful to note that use of anycast is impossible to hide. any looking glass will tell you that CF has tons of routes to each apparent IP, e.g. https://ssp.pme.gin.ntt.net/lg/lg.cgi?query=bgp&sourceIP=IP&..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Why and how is Hacker News so fast?"}},"_tags":["comment","author_Hello71","story_9990630"],"author":"Hello71","comment_text":"although it's useful to note that use of anycast is impossible to hide. any looking glass will tell you that CF has tons of routes to each apparent IP, e.g. https://ssp.pme.gin.ntt.net/lg/lg.cgi?query=bgp&sourceIP=IP&...","created_at":"2015-08-02T14:47:22Z","created_at_i":1438526842,"objectID":"9990892","parent_id":9990840,"story_id":9990630,"story_title":"Ask HN: Why and how is Hacker News so fast?","updated_at":"2024-09-19T22:13:51Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"midom"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"So sad I can't see or reply to that comment anymore - I'm the author. \nSorry, HN, my personal blog rarely gets that wide circulation, I should've done better job at introducing the context.
This was just one of many systems that thousands of facebook engineers build and they did not think of some edge cases. We do know how to write queries or to operate our databases, in general."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"On \u201corder by\u201d optimization"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://dom.as/2015/07/30/on-order-by-optimization/"}},"_tags":["comment","author_midom","story_9983374"],"author":"midom","comment_text":"So sad I can't see or reply to that comment anymore - I'm the author. \nSorry, HN, my personal blog rarely gets that wide circulation, I should've done better job at introducing the context.
This was just one of many systems that thousands of facebook engineers build and they did not think of some edge cases. We do know how to write queries or to operate our databases, in general.","created_at":"2015-08-02T03:14:23Z","created_at_i":1438485263,"objectID":"9989598","parent_id":9985763,"points":null,"story_id":9983374,"story_title":"On \u201corder by\u201d optimization","story_url":"http://dom.as/2015/07/30/on-order-by-optimization/","updated_at":"2023-09-07T03:21:16Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"andrew1"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"One of the big pains of my work day is our accounting system which, while being great at what it does well, can be abysmally slow to respond to queries (the simple question of how much of X do we own now, takes it around three minutes to answer...). We know from experience though that internally it can deal with up to three requests at a time without it slowing down. If I only had one thread then three requests (which block) would take me nine minutes to process, with three threads I can get all the results back in three minutes."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"'We are no longer as optimistic about removing the GIL completely. '"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://code.google.com/p/unladen-swallow/wiki/ProjectPlan#Global_Interpreter_Lock"}},"_tags":["comment","author_andrew1","story_998385"],"author":"andrew1","children":[998988],"comment_text":"One of the big pains of my work day is our accounting system which, while being great at what it does well, can be abysmally slow to respond to queries (the simple question of how much of X do we own now, takes it around three minutes to answer...). We know from experience though that internally it can deal with up to three requests at a time without it slowing down. If I only had one thread then three requests (which block) would take me nine minutes to process, with three threads I can get all the results back in three minutes.","created_at":"2009-12-16T15:31:47Z","created_at_i":1260977507,"objectID":"998932","parent_id":998783,"story_id":998385,"story_title":"'We are no longer as optimistic about removing the GIL completely. '","story_url":"http://code.google.com/p/unladen-swallow/wiki/ProjectPlan#Global_Interpreter_Lock","updated_at":"2024-09-19T16:53:52Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"morgante"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"> I've heard of people storing the queries that they were going to send to the database in a database row.
In fairness, that's not all that different from prepared statements..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I am Mt. Gox's first employee \u2013 AMA"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.reddit.com/r/Bitcoin/comments/3fe92x/im_ashley_barr_aka_adam_turner_the_first_mtgox/"}},"_tags":["comment","author_morgante","story_9988090"],"author":"morgante","comment_text":"> I've heard of people storing the queries that they were going to send to the database in a database row.
In fairness, that's not all that different from prepared statements...","created_at":"2015-08-02T00:47:23Z","created_at_i":1438476443,"objectID":"9989280","parent_id":9988551,"points":null,"story_id":9988090,"story_title":"I am Mt. Gox's first employee \u2013 AMA","story_url":"http://www.reddit.com/r/Bitcoin/comments/3fe92x/im_ashley_barr_aka_adam_turner_the_first_mtgox/","updated_at":"2023-09-07T03:21:16Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"NAFV_P"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Yesterday I finished a job in Cheltenham, England. The post was 50 weeks long. The commute to and back from work came to around two and a half to three hours. Often I would feel physically sick on the bus due to insufficient rest. I would go to work even though I felt unwell. This includes dealing with a trapped ulnar nerve which means my arm and shoulder ache all day, and is aggravated by using a laptop.
I pushed myself too hard, but this can occur when you are expected to work till you drop dead from exhaustion. This is what my first supervisor expected of me, because she thought I was a low life piece of shit. I had hardly any experience in insurance, especially claims which a lot of people in the building considered to be the sharp end of the business. Administration on a new claim was estimated on average to take around 90 minutes, it took me a day and a half. I was furnished with written procedure notes (about 40 pages worth), a laptop with no mouse and told to get on with it. I didn't consider this to be unfair, because this is the attitude I have experienced in most of my jobs. I had been in my new role for two days when I was given this task. All the thanks I got was a casual remark from my supervisor "Well, that's our Christmas bonus gone isn't it."
I was often shouted at in the office. Being an open plan space this behaviour was witnessed by about 80 other people. If my queries were not understood at first, the response I would get is "WHAT?!". When the day ended and the office was mostly empty, my supervisor's behaviour became very hostile.
The experience was incredibly stressful. My ulnar nerve issue worsened, so now it feels like I have a knife stuck in my little finger. Also I became so nervous about making a mistake (because my mistakes were over emphasised) that I worked at a rate about 4 times slower than my colleagues. Trust in my proof reading ability was lost, so I would check every single lexicon in a letter of correspondence and rereading my work nearly a dozen times. Some people would recognise this as OCD. I was considered incapable of performing my role, a bit offensive I might add.
I have a suspicion that I was treated like this partially because I am genetically male. My hope is to get a new job in the company's call center, and occasionally I would dress as a woman (or most of the time). I would like to see how I am treated by my ex supervisor/torturer while wearing a skirt, and how someone who is quite religious reacts to someone slowly coming out as transgender. I recently referred to her as a "fucking bitch" in a conversation with one of my other work colleagues. I regret that a lot, because (s)he isn't a real woman, and I'm more of a woman than she'll ever be."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Living in Switzerland ruined me for America and its lousy work culture"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://ukiahcommunityblog.wordpress.com/2015/07/23/living-in-switzerland-ruined-me-for-america-and-its-lousy-work-culture/"}},"_tags":["comment","author_NAFV_P","story_9987816"],"author":"NAFV_P","comment_text":"Yesterday I finished a job in Cheltenham, England. The post was 50 weeks long. The commute to and back from work came to around two and a half to three hours. Often I would feel physically sick on the bus due to insufficient rest. I would go to work even though I felt unwell. This includes dealing with a trapped ulnar nerve which means my arm and shoulder ache all day, and is aggravated by using a laptop.
I pushed myself too hard, but this can occur when you are expected to work till you drop dead from exhaustion. This is what my first supervisor expected of me, because she thought I was a low life piece of shit. I had hardly any experience in insurance, especially claims which a lot of people in the building considered to be the sharp end of the business. Administration on a new claim was estimated on average to take around 90 minutes, it took me a day and a half. I was furnished with written procedure notes (about 40 pages worth), a laptop with no mouse and told to get on with it. I didn't consider this to be unfair, because this is the attitude I have experienced in most of my jobs. I had been in my new role for two days when I was given this task. All the thanks I got was a casual remark from my supervisor "Well, that's our Christmas bonus gone isn't it."
I was often shouted at in the office. Being an open plan space this behaviour was witnessed by about 80 other people. If my queries were not understood at first, the response I would get is "WHAT?!". When the day ended and the office was mostly empty, my supervisor's behaviour became very hostile.
The experience was incredibly stressful. My ulnar nerve issue worsened, so now it feels like I have a knife stuck in my little finger. Also I became so nervous about making a mistake (because my mistakes were over emphasised) that I worked at a rate about 4 times slower than my colleagues. Trust in my proof reading ability was lost, so I would check every single lexicon in a letter of correspondence and rereading my work nearly a dozen times. Some people would recognise this as OCD. I was considered incapable of performing my role, a bit offensive I might add.
I have a suspicion that I was treated like this partially because I am genetically male. My hope is to get a new job in the company's call center, and occasionally I would dress as a woman (or most of the time). I would like to see how I am treated by my ex supervisor/torturer while wearing a skirt, and how someone who is quite religious reacts to someone slowly coming out as transgender. I recently referred to her as a "fucking bitch" in a conversation with one of my other work colleagues. I regret that a lot, because (s)he isn't a real woman, and I'm more of a woman than she'll ever be.","created_at":"2015-08-01T20:48:36Z","created_at_i":1438462116,"objectID":"9988611","parent_id":9987816,"story_id":9987816,"story_title":"Living in Switzerland ruined me for America and its lousy work culture","story_url":"https://ukiahcommunityblog.wordpress.com/2015/07/23/living-in-switzerland-ruined-me-for-america-and-its-lousy-work-culture/","updated_at":"2024-09-19T22:13:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nadams"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"> The entire codebase was stored in a database
Even if this didn't happen at Mt. Gox I'm sure it's happened in other places. I've heard of people storing the queries that they were going to send to the database in a database row."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I am Mt. Gox's first employee \u2013 AMA"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.reddit.com/r/Bitcoin/comments/3fe92x/im_ashley_barr_aka_adam_turner_the_first_mtgox/"}},"_tags":["comment","author_nadams","story_9988090"],"author":"nadams","children":[9988600,9988741,9989280],"comment_text":"> The entire codebase was stored in a database
Even if this didn't happen at Mt. Gox I'm sure it's happened in other places. I've heard of people storing the queries that they were going to send to the database in a database row.","created_at":"2015-08-01T20:26:51Z","created_at_i":1438460811,"objectID":"9988551","parent_id":9988232,"story_id":9988090,"story_title":"I am Mt. Gox's first employee \u2013 AMA","story_url":"http://www.reddit.com/r/Bitcoin/comments/3fe92x/im_ashley_barr_aka_adam_turner_the_first_mtgox/","updated_at":"2024-09-19T22:13:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"patio11"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Notable tidbits (most of which were already public record if you followed the story very closely) include:
+ Mt. Gox commingled depositor and corporate funds. (Corroborated in the Japanese version of their docs released by the bankruptcy trustee.) Many in the community believed that they had promised not to do this (true) and they this was impressively unprincipled (I lean against this understanding).
+ Mt. Gox had standards of engineering professionalism which were not what one would hope would prevail at a financial institution with $500 million in assets. True and previously reported. Specific examples include lack of a staging environment, source control, testing procedure, engineering leadership, auditing... it goes on. Also might suggest in future not putting all code into MySQL.
+ At one point all BTC deposited at Gox were accessible by VNCing into a particular box, which suggests that claims that Gox had cold storage secured by keys kept in diverse physical locations were not consistent with conventional interpretations of engineering reality. (The new information here is "VNC." It was previously known that at one instant in time Gox proved its reserves by moving all it's BTC in a single transaction. This could be done in a way consistent with what the community thinks "cold storage" should mean; the VNC bit militates against that understanding.)
+ Employees of Mt. Gox believed, on the basic of public evidence, that the firm was losing money even at the peak. (I'd be interested in seeing their math for this; that was not the conclusion I reached in a napkin calculation in mid-2013.)
+ Wages at Tibbanne (which provided 100% of employees for Gox) were consistent with prevailing wages in Tokyo startups i.e. scandalously low. 50% of employees earned less than $2k per month; director-level salaries in order of $4k, except for...
+ ... explicitly alleged looting of customer funds to support director lifestyles.
+ Management was unable or unwilling to answer basic queries regarding finances internally.
+ Gox's bus number was one, and that bus would have resulted in $500 million being unrecoverablu lost. Management, when asked about this, lied brazenly. (Last part is new info, first part obvious.)
There's more at the link."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I am Mt. Gox's first employee \u2013 AMA"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.reddit.com/r/Bitcoin/comments/3fe92x/im_ashley_barr_aka_adam_turner_the_first_mtgox/"}},"_tags":["comment","author_patio11","story_9988090"],"author":"patio11","children":[9988247,9988270,9988286,9988369,9988383,9988555,9988692,9989217,9995142],"comment_text":"Notable tidbits (most of which were already public record if you followed the story very closely) include:
+ Mt. Gox commingled depositor and corporate funds. (Corroborated in the Japanese version of their docs released by the bankruptcy trustee.) Many in the community believed that they had promised not to do this (true) and they this was impressively unprincipled (I lean against this understanding).
+ Mt. Gox had standards of engineering professionalism which were not what one would hope would prevail at a financial institution with $500 million in assets. True and previously reported. Specific examples include lack of a staging environment, source control, testing procedure, engineering leadership, auditing... it goes on. Also might suggest in future not putting all code into MySQL.
+ At one point all BTC deposited at Gox were accessible by VNCing into a particular box, which suggests that claims that Gox had cold storage secured by keys kept in diverse physical locations were not consistent with conventional interpretations of engineering reality. (The new information here is "VNC." It was previously known that at one instant in time Gox proved its reserves by moving all it's BTC in a single transaction. This could be done in a way consistent with what the community thinks "cold storage" should mean; the VNC bit militates against that understanding.)
+ Employees of Mt. Gox believed, on the basic of public evidence, that the firm was losing money even at the peak. (I'd be interested in seeing their math for this; that was not the conclusion I reached in a napkin calculation in mid-2013.)
+ Wages at Tibbanne (which provided 100% of employees for Gox) were consistent with prevailing wages in Tokyo startups i.e. scandalously low. 50% of employees earned less than $2k per month; director-level salaries in order of $4k, except for...
+ ... explicitly alleged looting of customer funds to support director lifestyles.
+ Management was unable or unwilling to answer basic queries regarding finances internally.
+ Gox's bus number was one, and that bus would have resulted in $500 million being unrecoverablu lost. Management, when asked about this, lied brazenly. (Last part is new info, first part obvious.)
There's more at the link.","created_at":"2015-08-01T18:53:43Z","created_at_i":1438455223,"objectID":"9988229","parent_id":9988090,"story_id":9988090,"story_title":"I am Mt. Gox's first employee \u2013 AMA","story_url":"http://www.reddit.com/r/Bitcoin/comments/3fe92x/im_ashley_barr_aka_adam_turner_the_first_mtgox/","updated_at":"2024-09-19T22:13:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"NicoJuicy"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Objects.Where(el => el.Id == myRequestedId).FirstOrDefault().Name; //Crashes when there is no result, but this has been handled by the null operator in c# 5.0 => Objects.Where(el => el.Id == myRequestedId).FirstOrDefault()?.Name; The issues was not Linq, it is/was c#
Single() is when there is absolute only 1 result. > 1 result is not expected ( i'll admit that i never use Single or SingleOrDefault() )
And using paging is the same thing in linq as it is in SQL. If you don't know how Linq works, you should learn it. What you should know is that .ToList() hits the database, so when you query all the objects, you get all the objects. Limit your query before the database query :)
Here are the results of page 4 with 10 items : Objects.Skip(10 x 3).Take(10).ToList(); //fetches 10 items = paging in the query
Here is how someone could write it without knowing Linq: Objects.ToList().Skip(10 x 3).Take(10);//fetches all the items and does the paging on the list.
As long as your DataType is an IQueryable<T>, it doesn't hits the database. When it's a List<T> (eg. when calling .ToList()) it's too late"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Java LINQ \u2013 C# LINQ examples running on Android"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/mythz/java-linq-examples/blob/master/README.md"}},"_tags":["comment","author_NicoJuicy","story_9986938"],"author":"NicoJuicy","children":[9987474],"comment_text":"Objects.Where(el => el.Id == myRequestedId).FirstOrDefault().Name; //Crashes when there is no result, but this has been handled by the null operator in c# 5.0 => Objects.Where(el => el.Id == myRequestedId).FirstOrDefault()?.Name; The issues was not Linq, it is/was c#
Single() is when there is absolute only 1 result. > 1 result is not expected ( i'll admit that i never use Single or SingleOrDefault() )
And using paging is the same thing in linq as it is in SQL. If you don't know how Linq works, you should learn it. What you should know is that .ToList() hits the database, so when you query all the objects, you get all the objects. Limit your query before the database query :)
Here are the results of page 4 with 10 items : Objects.Skip(10 x 3).Take(10).ToList(); //fetches 10 items = paging in the query
Here is how someone could write it without knowing Linq: Objects.ToList().Skip(10 x 3).Take(10);//fetches all the items and does the paging on the list.
As long as your DataType is an IQueryable<T>, it doesn't hits the database. When it's a List<T> (eg. when calling .ToList()) it's too late","created_at":"2015-08-01T15:50:29Z","created_at_i":1438444229,"objectID":"9987460","parent_id":9987407,"story_id":9986938,"story_title":"Java LINQ \u2013 C# LINQ examples running on Android","story_url":"https://github.com/mythz/java-linq-examples/blob/master/README.md","updated_at":"2024-09-19T22:13:31Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jameshart"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"It's better to write code that is amenable to being understood when read. These Java forms accomplish things 'the LINQ way' but at the expense of completely obscuring the intent of the code. LINQ makes sense in C# because the syntax supports a mapping from a declarative query to stream-oriented operations. Without the syntactic support, it becomes unreadable."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Java LINQ \u2013 C# LINQ examples running on Android"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/mythz/java-linq-examples/blob/master/README.md"}},"_tags":["comment","author_jameshart","story_9986938"],"author":"jameshart","children":[9988939],"comment_text":"It's better to write code that is amenable to being understood when read. These Java forms accomplish things 'the LINQ way' but at the expense of completely obscuring the intent of the code. LINQ makes sense in C# because the syntax supports a mapping from a declarative query to stream-oriented operations. Without the syntactic support, it becomes unreadable.","created_at":"2015-08-01T15:23:07Z","created_at_i":1438442587,"objectID":"9987341","parent_id":9987152,"story_id":9986938,"story_title":"Java LINQ \u2013 C# LINQ examples running on Android","story_url":"https://github.com/mythz/java-linq-examples/blob/master/README.md","updated_at":"2024-09-19T22:13:31Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"acqq"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Goldenkey actually posted
SELECT c,d FROM t WHERE a=(SELECT MAX(a) WHERE b=X from t)
But that post got flagged because he also gave his opinion on the level of understanding the article writer has. The query you posted is actually the one by the article writer."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"On \u201corder by\u201d optimization"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://dom.as/2015/07/30/on-order-by-optimization/"}},"_tags":["comment","author_acqq","story_9983374"],"author":"acqq","children":[9989102],"comment_text":"Goldenkey actually posted
SELECT c,d FROM t WHERE a=(SELECT MAX(a) WHERE b=X from t)
But that post got flagged because he also gave his opinion on the level of understanding the article writer has. The query you posted is actually the one by the article writer.","created_at":"2015-08-01T09:04:30Z","created_at_i":1438419870,"objectID":"9986424","parent_id":9986029,"story_id":9983374,"story_title":"On \u201corder by\u201d optimization","story_url":"http://dom.as/2015/07/30/on-order-by-optimization/","updated_at":"2024-09-19T22:13:26Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Animats"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Right. MySQL's optimizer knows how many records each table has, but not much else. It doesn't do statistical tests on the tables. Depending on table contents, "WHERE b=X" can select any number of records, from zero to the whole database. Whether the optimal strategy is to do the "WHERE b=X" first, possibly getting a large number of hits, or simply making a sequential pass, is not decidable from the database size only.
An alternative would be for the optimizer to do an indexed SELECT on b for "WHERE b=X", and if the number of hits exceeded some threshold, abandon that effort as inefficient and start over using a sequential pass. But MySQL doesn't abandon strategies once selected, or keep such statistics as a hint for future queries. As someone pointed out, Oracle's flagship database product does do that. Anyone know about Postgres?"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"On \u201corder by\u201d optimization"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://dom.as/2015/07/30/on-order-by-optimization/"}},"_tags":["comment","author_Animats","story_9983374"],"author":"Animats","comment_text":"Right. MySQL's optimizer knows how many records each table has, but not much else. It doesn't do statistical tests on the tables. Depending on table contents, "WHERE b=X" can select any number of records, from zero to the whole database. Whether the optimal strategy is to do the "WHERE b=X" first, possibly getting a large number of hits, or simply making a sequential pass, is not decidable from the database size only.
An alternative would be for the optimizer to do an indexed SELECT on b for "WHERE b=X", and if the number of hits exceeded some threshold, abandon that effort as inefficient and start over using a sequential pass. But MySQL doesn't abandon strategies once selected, or keep such statistics as a hint for future queries. As someone pointed out, Oracle's flagship database product does do that. Anyone know about Postgres?","created_at":"2015-08-01T07:05:22Z","created_at_i":1438412722,"objectID":"9986223","parent_id":9983374,"story_id":9983374,"story_title":"On \u201corder by\u201d optimization","story_url":"http://dom.as/2015/07/30/on-order-by-optimization/","updated_at":"2024-09-19T22:13:26Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"aikah"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"> The filesystem is a fantastic way to persist data for 80-90% of mobile applications.
Well Sqlite provides an extensive query language. I don't think Android's file system does."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Improving Facebook's Performance on Android with FlatBuffers"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/?utm_source=codedot_rss_feed&utm_medium=rss&utm_campaign=RSS+Feed"}},"_tags":["comment","author_aikah","story_9983339"],"author":"aikah","children":[9986033],"comment_text":"> The filesystem is a fantastic way to persist data for 80-90% of mobile applications.
Well Sqlite provides an extensive query language. I don't think Android's file system does.","created_at":"2015-08-01T04:52:55Z","created_at_i":1438404775,"objectID":"9985984","parent_id":9985562,"points":null,"story_id":9983339,"story_title":"Improving Facebook's Performance on Android with FlatBuffers","story_url":"https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/?utm_source=codedot_rss_feed&utm_medium=rss&utm_campaign=RSS+Feed","updated_at":"2023-09-07T03:20:56Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nostrademons"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Yes, most apps get a performance boost when they don't use a framework, and yes, more importance is placed on coders building products efficiently rather than coders building efficient products.
In most cases, this is the right choice. According to the TechEmpower benchmarks [1], a simple Django webapp can serve about 500 RPS on a c3.large EC2 instance, which will cost you about $54/month [2]. Adding an extra engineer will cost you about $15K/month (fully loaded). Therefore, for the cost of one engineer, you can pay for about 275 EC2 instances, which can serve 140K RPS, or about 12M hits/day.
Very few sites get that sort of traffic. Go work on speeding up the other bottlenecks in your app (like queries that lack indexes, or recursive queries), and don't make things harder on yourself than they need to be.
[1] https://www.techempower.com/benchmarks/#section=data-r10&hw=...
[2] http://aws.amazon.com/ec2/pricing/"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Are web frameworks actually useful?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":""}},"_tags":["comment","author_nostrademons","story_9985918"],"author":"nostrademons","children":[9987934],"comment_text":"Yes, most apps get a performance boost when they don't use a framework, and yes, more importance is placed on coders building products efficiently rather than coders building efficient products.
In most cases, this is the right choice. According to the TechEmpower benchmarks [1], a simple Django webapp can serve about 500 RPS on a c3.large EC2 instance, which will cost you about $54/month [2]. Adding an extra engineer will cost you about $15K/month (fully loaded). Therefore, for the cost of one engineer, you can pay for about 275 EC2 instances, which can serve 140K RPS, or about 12M hits/day.
Very few sites get that sort of traffic. Go work on speeding up the other bottlenecks in your app (like queries that lack indexes, or recursive queries), and don't make things harder on yourself than they need to be.
[1] https://www.techempower.com/benchmarks/#section=data-r10&hw=...
[2] http://aws.amazon.com/ec2/pricing/","created_at":"2015-08-01T04:49:28Z","created_at_i":1438404568,"objectID":"9985973","parent_id":9985918,"points":null,"story_id":9985918,"story_title":"Are web frameworks actually useful?","story_url":"","updated_at":"2023-09-07T03:20:56Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"schrodinger"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["query"],"value":"Yea, the guy that goes into the source code of the optimizer to figure out the cause for a poorly optimized query plan is the amateur!"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"On \u201corder by\u201d optimization"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://dom.as/2015/07/30/on-order-by-optimization/"}},"_tags":["comment","author_schrodinger","story_9983374"],"author":"schrodinger","children":[9989598],"comment_text":"Yea, the guy that goes into the source code of the optimizer to figure out the cause for a poorly optimized query plan is the amateur!","created_at":"2015-08-01T03:17:07Z","created_at_i":1438399027,"objectID":"9985763","parent_id":9985069,"points":null,"story_id":9983374,"story_title":"On \u201corder by\u201d optimization","story_url":"http://dom.as/2015/07/30/on-order-by-optimization/","updated_at":"2023-09-07T03:20:56Z"}],"hitsPerPage":50,"nbHits":145167,"nbPages":20,"page":0,"params":"query=%7Bquery%7D&tags=comment&hitsPerPage=50&advancedSyntax=true&analyticsTags=backend","processingTimeMS":18,"processingTimingsMS":{"_request":{"roundTrip":14},"afterFetch":{"format":{"highlighting":3,"total":3},"merge":{"mergeLoop":{"prepareNextHit":6,"total":6},"total":6},"total":6},"fetch":{"query":3,"scanning":6,"total":10},"total":18},"query":"{query}","serverTimeMS":22}