When I posed the original question, what I meant to ask was "What recognition-based tools are there that we can use for search?"
Here's our provisional list so far. These are all apps (or tools) you can use to recognize objects or signals in the world.
I've grouped them by sensing mode (that is, how does your query device sense the signal it's going to try and recognize). When you think about it that way, all of the "image reco" apps go together (they're all variations on image processing) and all of the "audio reco" apps go together.
What sense modalities are we missing here?
Obviously, there's no taste reco system (or is there one and I don't know about it??) Wouldn't it be interesting if there were a wine varietal recognition tool. (Pour a little of your plonk onto your cell phone--have it identify the variety, terrior, and year!)
Remember--our goal in this is to increase the range of tools we know about that help us be more effective searchers.
Are there more reco-based search tools that we're missing from this list?
Here's our provisional list so far. These are all apps (or tools) you can use to recognize objects or signals in the world.
I've grouped them by sensing mode (that is, how does your query device sense the signal it's going to try and recognize). When you think about it that way, all of the "image reco" apps go together (they're all variations on image processing) and all of the "audio reco" apps go together.
What sense modalities are we missing here?
Obviously, there's no taste reco system (or is there one and I don't know about it??) Wouldn't it be interesting if there were a wine varietal recognition tool. (Pour a little of your plonk onto your cell phone--have it identify the variety, terrior, and year!)
Remember--our goal in this is to increase the range of tools we know about that help us be more effective searchers.
Are there more reco-based search tools that we're missing from this list?
Search on (for more search tools...)!
Vision
Google Goggles Runs
augmented Image search of scene.
LeafSnap Recognizes
deciduous leaves of North American trees.
LookTel Money; cans
of food. Intended for low vision users to find everyday items.
Evernote Part of Evernote
does handwritten text recognition.
Recognizes
hand-written Chinese, Japanese, Arabic, etc.
Picasa / iPhoto Recos
faces from the set of faces you’ve tagged .
WhatTheFont Identifies the font in
which a given text is set in an image.
IdMyPill Identifies pills by
size,shape, color.
Project Shepherd Identifies individual animals
out of a tagged collection by
their particular spot pattern. Open source project.
Requires some setup to use. Works for whale sharks,
their particular spot pattern. Open source project.
Requires some setup to use. Works for whale sharks,
manta rays, etc.
Searches for
images based on similarity to images on the net.
ColorID
ColorReader
gPick Identifies a specific
region of an image by color name
(e.g., “teal”
or “plum”)
PDFpen Scan+
Tesseract (open source)
Pixter Number of character recog
tools. Point at an image of text and
recover the
the text itself by recognition. (Which you can then search via Control-F.)
the text itself by recognition. (Which you can then search via Control-F.)
StarMap Augmented Reality apps. Point your phone at a position in the sky and
see what's there.
Audio
Soundhound Identify pre-recorded
music (not live performance)
WeBird Identify bird songs (Android)
SongTapper Recognize (some)
popular songs by tapping the rhythm on the space bar
(It worked
rarely for me…)
Viggle Identifies TV show you’re
watching by recognizing the sound track.
Musipedia Identify music by
whistling (difficult to make work), or playing on
a keyboard.
a keyboard.
TunePal Identify
live performances of Irish Session tunes.
GPS / Location sensing
FieldTrip
Looks for
interesting historical sites near your current location
(US
only?)
Rutísima
Madre Looks
for interesting historical sites near your current location
(Mexico
only?)
GeoGoogle
GPS
augmented info.
Peaks
Mountain
peak ID by GPS + camera
NokiaCity Lens Identify buildings
in city by AR on cell phone
I would call Field Trip international. Looking through the categories they have just for architecture and historic events, I see things for Dublin, Historic London, Berlin and more. The cool and unique category includes items from Atlas Obscura that appears to be a worldwide travel database http://www.atlasobscura.com/ .
ReplyDeleteAlthough olfactory (smell / odor) recognition appears to be technically less difficult than taste recognition, I don't know of any working e-nose for smartphone.
ReplyDeleteAnother modality (different from location sensing, but probably best working together) is kinetic (movement and balance) recognition. This might include gesture recognition (movement outside the recognition device or on the touch screen) and proprioception (device movement recognition, using a gyroscope and / or signal location).
Still another modality would be thermoception (temperature recognition, thermometer). I don't know of any smartphone with this capability. Instead, they use data from a web service after locating you.
Other modalities can be thought of. Check the Wikipedia article on Sense.
Then, within image recognition, there's facial recognition.
Still related to image recognition, before Google Search by Image was any good, I used the excellent TinEye Reverse Image Search a lot. I liked it so much I was sure Google would buy it (only it didn't). :)
:( GeoGuessr
Deletedon't know if this is still in the pipeline, but would seem to fall in the '"kinetic" category you defined as well as language translation by image/symbol/sign.
ReplyDelete☞language translator
It definitely fits. Query: [ what is that person saying in ASL ]
Deletefailed to include:
ReplyDeleteTechnabling
PSLT
Interesting. The Technabling is a sensor net that detects abnormal patterns of behavior--a neat app, but not "search-y" in the normal sense. Fascinating, tho.
DeleteSo I went down this rabbit hole [ how many senses do human have ] (I figured I give hummingbird semantic search a try) and got to Humans Have a Lot More Than Five Senses
ReplyDeleteBack to searching for apps and/or web sites for reco.
Fred -- It's true--we do have many more than 5 senses. (Likewise, computers have many more senses than we do. For instance, measuring the number of disk errors can be an indicator of earthquakes--see: http://www.eduard-heindl.de/IEEE-p2p-heindl.pdf )
ReplyDeleteIt would be interesting to figure out what "sense" computers really DO have!
Dan, not to be overly risqué in response, but this is sign language related - as well as art and computer manufactured and tech related and your daughter's art class — yesterday, in Prague… not a text or tweet, but still a bird… of sorts - hope there isn't a test to trace it to the Etruscans.
ReplyDeletemessage in Prague
DAVID ČERNÝ
from London 2012
Wikipedia entry
Kind of off topic.
ReplyDeleteReading my streamline on Google Plus found Handwriting input comes to Gmail and Google Docs It is not an app; and, it is a sequel from the What does this character mean?
Have a great day. See you tomorrow in a new Search Research Challenge
GPS / Location sensing
ReplyDeleteMost of the navigation apps begin with identifying your location first, then helping you find directions to a destination. Most allow you to either type in your destination address or search for it. Like Field Trip where you aren't searching for something specific, these apps can also offer different types of places near your location such as stores, hotels, gas stations and more.
Three apps that I use regularly on iOS (also available on Android)
Google Maps Mobile
Waze
Navigon
Staying with the GPS/Location sensing apps
LocalScope (iOS) is again similar to Field Trip in that it is a location browser. Based on your location, Localscope can help you find Google+ reviews, Twitter tweets, Flickr photos, Youtube videos and many more. You can view the results in a list, on a map or through the camera in an augmented reality view. Localscope integrates with the 3 apps above and more for directions. Localscope supports 21 languages.
Similar to Localscope
AroundMe (iOS, Android, Windows Phone)
An occasional hobby of mine is Geocaching. For those unfamiliar with geocaching, it is a type of treasure hunt based on GPS location. You are provided coordinates for a hidden cache. Using your GPS device you hunt for cache using your position and the position of the cache. The Geocaching app (iOS, Android, Windows Phone) allows you to find nearby caches based on your location wherever you are. Then as you search for the cache the app constantly updates based on your location. Searching IRL!
Another one is Bar Code Scanner, and other bar code scanning tools
ReplyDeleteIntoNow is a great audio app that can identify a TV show and episode with just a few seconds of audio.
ReplyDeleteWhile looking at optical character recognition apps, I remembered that there are cheating apps for apps like Words With Friends and Hanging With
ReplyDeleteFriends. You take a screenshot of your game screen and then uploaded to the cheating app. You are then presented with a list of possible words
to play. That reminded me of a post that taught me how to cheat at Words With Friends using Wolfram Alpha. The author of the post stated "Wolfram Alpha is a remarkable resource that's chock full of ways to find out
about science and math."
Thinking of that post, Wolfram Alpha is recognizing letters and spaces (blanks) and then identifying words that fit the pattern. Example Re_earch. Keywords that we have been using are recognizing and then identifying.
I wondered if Wolfram Alpha could recognize and then identify other patterns. I found that Wolfram Alpha can recognize and identify number sequences [ 1,1,2,3,5,8,13,...
]as well as recognize and identify genealogic relationships [ mother's father's sister's daughter's son ].
Wolfram Alpha (iOS, Android, nook, Kindle Fire)
Now I know this gets away from the framework of human senses, it is
still "a remarkable resource."
Aurasmas - see this video that explains the software www.youtube.com/watch?v=JrMxVmvBAVg
ReplyDeleteWhile it may be used as a marketing tool because anyone can create an "aura" it can be used to bring print to life. I can see all kinds of applications. What I couldn't figure out is how to use it to create an aura using an online object.
I can understand the 'recognition' app to a certain point. If its geographically related than unless we can simulate the location searching isn't possible. On a Garmin you can simulate the coordinates ( I use it if creating a route for a hike using my Garmin). What would be really cool is if we could use Street View.
Two other apps that I have very little knowledge of is Layar and Junaio. I'm not sure if these are any better than say Google Earth POI's but they provide a mobile platform.
I'm missing Layar and Wikitude in this collection.
ReplyDeleteHere's a link to investigative tools used by Journalists that links to sites we could use so this goes under 'tools'.
ReplyDeleteI know we have used it before but another tool that is expanding is the "Layers" in Google Earth. I see they now have a link to historical maps and of course National Geographic but many more.
I may have forgotten to give the link to The Journalist's Toolbox
ReplyDeletehttp://www.journaliststoolbox.org/archive/investigative/
Thanks Rosemary, The Journalist's Toolbox is a treasure trove - additional suggestions from there:
DeleteGannett; 14 Best…
as everything is becoming a mobile space, this might be the quickest way to keep up with the updates…
JT on twitter
an indication of how quickly things change; this was a featured site on the J'toolbox box… R.I.P, Scirus -
scirus, gone in 2014
oarfish search example:
Oarfish
from the scirus site - pre note keeping apps -
new scientist
Google+ Computer Vision Search - began in May and was expanded to more than 1,000 objects this week. Search Google+ photos for terms like [ labrador ] [ food ] and more.
ReplyDeleteWondering if the Computer Vision Search is related to NEIL (Never Ending Image Learner)
DeleteI saw this post today by Google in Education. If the Smart Measure tool works as advertised then Smart Tools gets added to the list.
ReplyDeleteTwo more:
ReplyDeleteGet your next eye exam on a smartphone
This smartphone app can detect skin cancer - this appears to do what we through the U of M skinscan app did, but requires a $500 lens to do it.
Jetpac offers two apps for iOS.
ReplyDeleteDeep Belief allows you to teach your phone to recognize objects.
Spotter allows your iPhone camera to identify objects. I tired this one and it was able to identify a few things in my office accurately. I haven't figured out how useful this is to the average person. Does anyone have any ideas?
I can see where the usage data from the teacher app might be used in a project mentioned above such as Google's Computer vision, but on a day to day use I can't think of how this would be helpful.
Store receipt scanning apps seem to be on the rise. A couple of apps I've been using are Snap and Checkout51. They are basically electronic coupon apps. You tell the app what you bought and upload a picture of your receipt. They verify your purchase and credit you money for your purchase.
ReplyDeleteA new one I just found today is CookBrite. Unfortunately, it is Android only so I can't try it out, but the concept is very useful. You can keep a record of what you have on hand by snapping a picture of your receipt. You can then search for recipes based on what you have on hand.