tag:blogger.com,1999:blog-54222021814977934802024-03-18T20:26:12.494-07:00Albo WanderingsAnonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.comBlogger17125tag:blogger.com,1999:blog-5422202181497793480.post-10758095651588702592015-12-11T02:01:00.001-08:002015-12-11T02:01:52.880-08:00Creating Elevation Profiles from Rasters in ArcGISAs with many things GIS, there are a couple tricks you need to know if you want to generate an elevation profile. In my case, I'm using ArcGIS and its 3D Analyst extension to generate elevation profiles using SRTM DEM data (see <a class="moz-txt-link-freetext" href="http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp">http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp</a> for data download - I chose to get the data in "ArcINFO ASCII" format).<br> <br> Getting the DEM into ArcGIS is a no-brainer. Just download the data, unzip it, and use ArcCatalog to add it to map.<br> <br> To use 3D Analyst to generate the profile, you have to enable the 3D Analyst Extension, and then add the 3D Analyst toolbar (right click in an empty area of your toolbar and click 3D Analyst). Then you can draw a 3D polyline (click the <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgq1KlX3plw0sjhkAjcnd4GmtIb_2mTEWkXu2XRhB7x9i5PzPvoLJAst7u6tbSfyPx-8Cqttgrl6w-HSOGB-5cSpLkJCsmdHHGyeg8oNUuokWy74bXb1a_OT_5PIOHOc1R4JFQV1o4advaF/s1600/cfbaiigh-712881.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgq1KlX3plw0sjhkAjcnd4GmtIb_2mTEWkXu2XRhB7x9i5PzPvoLJAst7u6tbSfyPx-8Cqttgrl6w-HSOGB-5cSpLkJCsmdHHGyeg8oNUuokWy74bXb1a_OT_5PIOHOc1R4JFQV1o4advaF/s320/cfbaiigh-712881.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6226964327654060098" /></a> button) along the path for which you want the profile. Then, click the profile graph button (<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZtC1rkkG87ectBZQ64Dg_Ihg-rt0gWEIL-palVGGXsEtw46es6PaW28kFh-HkixSlh5Xah88xcgl3f1owlVVKRCbkMdMpZD7hQ1-dXGQRb1u0So4En4cnEpLcmcYGZ3hOHJ6wJ4BjJ9K4/s1600/heiebbjb-713748.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZtC1rkkG87ectBZQ64Dg_Ihg-rt0gWEIL-palVGGXsEtw46es6PaW28kFh-HkixSlh5Xah88xcgl3f1owlVVKRCbkMdMpZD7hQ1-dXGQRb1u0So4En4cnEpLcmcYGZ3hOHJ6wJ4BjJ9K4/s320/heiebbjb-713748.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6226964332237919650" /></a>) to generate the profile. The data is saved where you've configured it under the 3D Analyst -> options menu (<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2ljrjF_viw7Wj-65zQfd3x-DTMUixoKwuhxIrXeJ8Utn1dCOgNDwDbOqEBEMkCgvR0ovQrbjvWSKfmjWjqdsHnoUiYd_l5nIsywLVBDg-zAy3786zaUbNX84lflosfcLQ6wfZTWCmtXiY/s1600/fadjbica-714687.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2ljrjF_viw7Wj-65zQfd3x-DTMUixoKwuhxIrXeJ8Utn1dCOgNDwDbOqEBEMkCgvR0ovQrbjvWSKfmjWjqdsHnoUiYd_l5nIsywLVBDg-zAy3786zaUbNX84lflosfcLQ6wfZTWCmtXiY/s320/fadjbica-714687.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6226964337374167010" /></a>).<br> <br> But I found an unpleasant surprise - my resulting dbf's (you can open them with excel) were reporting a <b>stretched </b>value - not the actual elevation value from the SRTM data. So, here's the trick: you have to set the "Source Type" of your SRTM data to "Elevation". To do that, right-click on your elevation raster in ArcCatalog, find the "Source Type" row, and switch it to "Elevation". Now, re-add that DEM to your map, draw your 3D line again, generate your profile again, and voila - the dbf should now be reporting actual elevation and not a stretched value.<br> <br> I didn't find this information anywhere else, so I'm hoping this might help some folks in the same situation I found myself in.<br> Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-22860022680562487542014-08-19T03:39:00.001-07:002014-08-19T03:39:57.005-07:00How to Keep Google from Profiling YouI just sent this email to my brother, who is interested in not being profiled by google. I thought I would share it with you all as well. This is a list of firefox extensions to consider if you want to keep google (and other bad boys) from following you everywhere. Note that you <i>don't </i>need admin rights on your computer to install these.<br> <br> To hide yourself from google, i would suggest doing a few things, which will be good for all your stuff, actually:<br> <ol> <li>Install CS Lite Mod - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/cslite-mod/">https://addons.mozilla.org/en-US/firefox/addon/cslite-mod/</a></li> <ol> <li>Set it to Deny Cookies Globally, and then allow cookies on a site-by-site basis. You may occassionally have sites act shitty on you, in which case you can allow cookies for them that will persist forever, you can allow cookies per session (which allows that site to set cookies, but times them out after a while), or you can temporarily allow cookies, which turns that site back into a "no cookie site" after a while.</li> <li>Allow google to set cookies on a per-session basis<br> </li> </ol> <li>Install Adblock Plus - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/?src=search">https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/?src=search</a></li> <ol> <li>Enable the ad filters, tracking filters, and whatever else you want</li> <li>Ad the following lines so that google won't follow you around the <br> </li> </ol> <li>Install Google Redirects Fixer - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/google-no-tracking-url/?src=search">https://addons.mozilla.org/en-US/firefox/addon/google-no-tracking-url/?src=search</a></li> <ol> <li>You know how when you click on a google search result, it first shows a google URL in your address bar, and then forwards you to the site? Yeah, that's google tracking what you click. This addon will make google search links link directly to the site you want, and not through a google redirect.</li> </ol> <li>Install RefControl - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/refcontrol/">https://addons.mozilla.org/en-US/firefox/addon/refcontrol/</a></li> <ol> <li>When you click on a link, your browser sends a query to the site at that link location to send you the page. Along with the query, it tells that site where you're coming from. So, if you click on a link that sends you to google from a lifehacker page, google will know you're coming from lifehacker. RefControl will allow you to stop your browser from sending that information, and it will allow you to turn it back on on a site-by-site basis (if stripping that ref info breaks a site for example - but it usually never does).</li> </ol> <li>Install DoNotTrackMe - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/donottrackplus/">https://addons.mozilla.org/en-US/firefox/addon/donottrackplus/</a></li> <ol> <li>I don't use this myself, but I probably should. As I understand it, it keeps your browser from alerting tracking companies that you're on a particular page. I don't use it because I have CSFire installed (see below). But this might be better.<br> </li> </ol> <li>If you really want to fuck with google, install TrackMeNot - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/trackmenot/">https://addons.mozilla.org/en-US/firefox/addon/trackmenot/</a></li> <ol> <li>It will send random search strings to google, thereby obfuscating their user profile for you</li> </ol> <li>You might think about installing NoScript - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/noscript/">https://addons.mozilla.org/en-US/firefox/addon/noscript/</a></li> <ol> <li>This will block lots of nefarious tracking shite. But it will also break lots of sites. So, I don't use it. But you might think about it.</li> </ol> <li>I use a thing called CsFire - <a class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/csfire/">https://addons.mozilla.org/en-US/firefox/addon/csfire/</a></li> <ol> <li>It's kind of a pain, so i don't really recommend you use it. But what it does is strip queries to third party content of cookies and the like. For example, if a lifehacker page is showing an image from google images, CSFire will allow you to grab the image, but it won't allow google to see your google-related cookies, etc. So, it will be harder for google to know that you're visiting the lifehacker page.</li> </ol> </ol> Ok, that's a lot. But do it!<br> <br> Feel free to (in fact, please do) share this with your colleagues or anyone else who wants to avoid the REAL big brother these days. (ok, that ended up sounding a little militia-like paranoid, but you know what I mean).<br> Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-56339661080984058182013-03-20T12:04:00.001-07:002013-03-20T12:18:18.308-07:00resolving factor frustration in R<div class="moz-text-plain" lang="x-western" style="font-family: -moz-fixed; font-size: 14px;" wrap="true">
<pre wrap=""><span style="font-family: "Trebuchet MS",sans-serif;">Factors in R have two components: their "index" (my term), which is just a vector of integers, and their "level", which is a vector of characters (or integers, though it's most helpful to think of them as just characters for now). Matloff explains this well in his book <a href="http://www.amazon.com/gp/product/1593273843/ref=as_li_ss_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=1593273843&linkCode=as2&tag=shenkinorg-20">The Art of R Programming</a><img alt="" border="0" class="bxldfrybhjpmtgeihzuk" height="1" src="http://www.assoc-amazon.com/e/ir?t=shenkinorg-20&l=as2&o=1&a=1593273843" style="border: none !important; margin: 0px !important;" width="1" />, but I'll give my own quick example here.</span>
<span style="font-family: "Courier New",Courier,monospace;">
> f = factor(c(10,11,12,11))
> f
[1] 10 11 12 11
Levels: 10 11 12
> levels(f)
[1] "10" "11" "12"
> unclass(f)
[1] 1 2 3 2
attr(,"levels")
[1] "10" "11" "12"
> attributes(f)
$levels
[1] "10" "11" "12"
$class
[1] "factor"</span>
<span style="font-family: "Trebuchet MS",sans-serif;">You can see that R's internal representation of factors is just as I explained above. That is, there is an integer index that is essentially a lookup table with integer indicies that point to the "labels" character vector. The "labels" character vector attribute is the factor's second component.
While perhaps a bit confusing, this is all relatively straightforward. Where factors can get really frustrating, however, is when different functions use different parts of the factor; some use the de-referenced "levels" of the factor, whereas others use the index values. This problem is most evident in the as.character() and as.numeric() functions:</span>
<span style="font-family: "Courier New",Courier,monospace;"> > as.character(f)
[1] "10" "11" "12" "11"
> as.numeric(f)
[1] 1 2 3 2</span>
<span style="font-family: inherit;">
<span style="font-family: "Trebuchet MS",sans-serif;">What's happening here? as.character() is dereferencing the index and returning the character levels. On the other hand, as.numeric() is returning the <i class="moz-txt-slash"><span class="moz-txt-tag">/</span>index vector<span class="moz-txt-tag">/</span></i> part of the factor. This is probably not what you'd expect if you have numeric factors, such as we have above. When you want to turn your factor into numbers, what you probably want to do is:</span></span>
<span style="font-family: "Courier New",Courier,monospace;"> > as.numeric(as.character(f))
[1] 10 11 12 11</span>
<span style="font-family: "Trebuchet MS",sans-serif;">
I've been using R for years now, and I finally took the time to look into this and resolve it. I'm sure I'll run into more unexpected behaviors in the future, but I'm glad to have at least solved this aspect of factor behavior in R. Do you have any other examples? Feel free to share them in the comments!</span>
</pre>
</div>
Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-52826917417641953122012-10-23T12:32:00.001-07:002012-10-26T09:02:08.334-07:00Importing Excel data in R with XLConnectI recently had to migrate to the 64-bit version of R due to the 4 GB memory limit imposed by the 32-bit version. In doing so, the package I was using for importing Excel files (xlsReadWrite) was rendered incompatible. Thus, I've moved over to XLConnect: <a class="moz-txt-link-freetext" href="http://cran.r-project.org/package=XLConnect">http://cran.r-project.org/package=XLConnect</a><br />
<br />
XLConnect looks very nice: it is fully featured, and is implemented as objects. It uses a Java library to do the Excel imports, apparently. This would seem like a detail, but it ends up adding a very important wrinkle. According to the developer, Java's "ToString" function adds a ".0" to any values that look numeric. So, even if you tell XLConnect's readWorksheet function to read the column in as a character data type (using the colTypes argument), it will add a ".0" to every value that looks to Java like a number.<br />
<br />
Here's an example. Let's make an excel document called XLConnect_test.xlsx:<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_KG_-oXCF-XS42x_qOue10WfwSZPxQ3654IxjMrJLPzScU8WMUkBRWpqmL2U_zoA5NyiF7Kp2KSbQ1VLQKtKbG23f9I_fiASZg_WHNLoYSbrBi3u6kD1LKsCzdHd82o-fdMOvaGo02idp/s1600/dadiaied-759604.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5802589984481630034" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_KG_-oXCF-XS42x_qOue10WfwSZPxQ3654IxjMrJLPzScU8WMUkBRWpqmL2U_zoA5NyiF7Kp2KSbQ1VLQKtKbG23f9I_fiASZg_WHNLoYSbrBi3u6kD1LKsCzdHd82o-fdMOvaGo02idp/s320/dadiaied-759604.png" /></a><br />
<br />
Now, let's try to import it via XLConnect:<br />
<blockquote>
<tt>> require(XLConnect)</tt><br />
<tt>> xlc_df = readWorksheetFromFile(file="XLConnect_test.xlsx", sheet=1)</tt><br />
<tt>> xlc_df</tt><br />
<br />
<tt> Col1 Col2</tt><br />
<tt>1 ABC 1.0</tt><br />
<tt>2 123.0 2.0</tt><br />
<tt>3 123ABC ABC</tt><br />
<br />
<tt>> xlc_df = readWorksheetFromFile(file="XLConnect_test.xlsx", sheet=1, colTypes=c("character", "character"))</tt><br />
<tt>> xlc_df</tt><br />
<br />
<tt> Col1 Col2</tt><br />
<tt>1 ABC 1.0</tt><br />
<tt>2 123.0 2.0</tt><br />
<tt>3 123ABC ABC</tt></blockquote>
Notice how XLConnect adds ".0" to each value, even though those columns are supposed to be character strings? That's not good. The nicely-communicative developer says that they're working on this for a future version. In the meantime, here's the hack I've written so the rest of us can move forward:<br />
<blockquote>
<tt>xlc_dot_zero_hack <- function(df) {</tt><br />
<tt> # looks for all numerics that had a ".0" added to them by XLConnect, and removes the ".0"</tt><br />
<tt> return(</tt><br />
<tt> as.data.frame( lapply( df, FUN = function(x) sub("^(\\d*)\\.0$", "\\1", x, perl=TRUE) ) )</tt><br />
<tt> )</tt><br />
<tt>}</tt></blockquote>
Just import your Excel data, and then run the function above on your imported dataframe. Don't forget to reassign the result to a dataframe. In our example, we would write:<br />
<blockquote>
<tt>> xlc_df = xlc_dot_zero_hack(xlc_df)</tt><br />
<tt>> xlc_df</tt><tt><br /> <br /> </tt><tt> Col1 Col2</tt><tt><br /> </tt><tt>1 ABC 1</tt><tt><br /> </tt><tt>2 123 2</tt><tt><br /> </tt><tt>3 123ABC ABC</tt></blockquote>
Voilà! Note that if your data has any strings that *actually* have a ".0" at the end, this hack will have unexpected results. Specifically, it will remove *all* ".0"s that come after a string of digits.<br />
<br />
I hope this might help others out there with the same conundrum!<br />
<br />
<br />
<a href="https://plus.google.com/102945187007367271699?rel=author”>Google + Profile</a>Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-34613978094843826912012-09-05T11:46:00.000-07:002012-09-05T11:47:13.254-07:00Cross-References don't show up in Word's dialog box?I was having an issue with a manuscript I'm putting together recently. I was passing it back and forth with my advisor, and I was using Track Changes. As I added some figures with captions, I then wanted to refer to those figures in the text via a cross-reference.<br> <br> When I clicked on Insert -> Cross-Reference in Word 2010, the recently-inserted captions weren't in the list!<br> <br> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuyYB4E5Whq-Cm5de_6GONNnnjqEWSymyXTyO_GyB_mJD89L2NmsrFOiMkDDiyPqDrRg33LFvEZmHOwUME0PLA0u96WynqMObnZaV8r08lqZVgq-7UjOuKj8sjcR2-OlYmkXHedEB_P_o8/s1600/fjcdgeij-733255.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuyYB4E5Whq-Cm5de_6GONNnnjqEWSymyXTyO_GyB_mJD89L2NmsrFOiMkDDiyPqDrRg33LFvEZmHOwUME0PLA0u96WynqMObnZaV8r08lqZVgq-7UjOuKj8sjcR2-OlYmkXHedEB_P_o8/s320/fjcdgeij-733255.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5784766182343734738" /></a><br> <br> Ugh, *another* Word screwup with complex documents.<br> <br> Well, after a lost day of screwing around with the problem (and sleeping on it), I came across the solution.<br> <br> The problem seems to be that Word doesn't like you to insert your figures and captions as a "tracked change". I went back and "accepted" all my insertions of figures and captions. And, voila! The figures now show up in the cross-reference dialog box. Now, whenever I insert a new figure, I'll either turn off track changes for the moment, or just go back and accept the insertion.<br> <br> Hope this helped somebody else!<br> Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-33504377620891008462012-05-09T08:24:00.000-07:002012-05-09T08:25:11.666-07:00pushd / popd for RHi Folks,
<br />
<br />
I haven't gotten around to putting my R code in nice objects yet. So, my code is procedural, and it's nasty. It sources one script to the next, processing data, analyzing data, etc. As the scripts get run around the codebase, they often change their working directory. So, I wanted access to a bash-equivalent pushd and popd. I'll run these at the beginning and end of each script.<br />
<br />
These are not battle-tested yet, and there's really no error-checking or redundancy in them. So, use with care, and they'll probably evolve over time. Feel free to add to them in the comments!<br />
<br />
Enjoy!<br />
<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
# push getwd() strings into a FIFO vector<br />
<br />
dir_fifo = c()<br />
<br />
pushd <- function(cd) {</div>
<div style="font-family: "Courier New",Courier,monospace;">
# usage: pushd("directory to change to")<br />
dir_fifo = append(dir_fifo, getwd(), 0)<br />
assign("dir_fifo", dir_fifo, envir = .GlobalEnv)<br />
setwd(cd)<br />
}<br />
<br />
popd <- function() {</div>
<div style="font-family: "Courier New",Courier,monospace;">
# usage: popd()<br />
setwd(dir_fifo[1])<br />
dir_fifo = dir_fifo[-1]<br />
assign("dir_fifo", dir_fifo, envir = .GlobalEnv)<br />
}</div>Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-9491574207155659882012-03-30T07:44:00.001-07:002012-03-30T07:44:36.699-07:00Do you miss seeing your Lenovo drivers ordered by release date?IBM used to have a page where you could view all the drivers for your<br>system ordered by their release date. Since the IBM/Lenovo "System<br>Update" app has always been pretty unreliable, seeing the driver<br>releases ordered by date was always (for me, anyway) the best way to<br>make sure my system was up to date.<p>Lenovo, in all their wisdom, got rid of the ability to order drivers by<br>release dates. But, thankfully, there's a workaround!<p>Here's what you do. Go to the Lenovo support site and enter your system<br>type here: <a href="http://support.lenovo.com/en_US/downloads/default.page">http://support.lenovo.com/en_US/downloads/default.page</a> . <br>Then, click on the "Subscribe to driver updates" link in the upper-right<br>corner. That will take you to an RSS page. Don't subscribe to it if<br>you don't want to! But from here, you'll be able to see all the driver<br>releases ordered by date. Voilá!Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-5321762737178746262012-03-07T05:35:00.003-08:002012-03-07T05:47:24.216-08:00Uninstalling HP's SmartPrintButton Extension from FirefoxI detest printer drivers. Well, not so much the drivers themselves, but all the useless junk that always comes with them. Sometimes you can find drivers that come separately from the entire software suite, but more often than not, you're stuck installing some sort of crap in order to be able to print to your new printer. Ugh.<br /><br />Case-in-point: I just bought an HP Laserjet 1606dn. I chose the absolute minimum installation necessary in order to print to the thing. But, I still ended up with a "SmartPrintButton" extension, uninvited, in my firefox add-on panel. Ugh (again)!<br /><br />Even worse, this extension was "locked" - i.e., there was no "remove" button I could click to get rid of thing. I tried removing it by starting firefox in safe mode, but that didn't get the job done.<br /><br />Finally, I went to my firefox extensions folder (C:\Users\username\AppData\Mozilla\Firefox\yourprofile\extensions) to track down the culprit and exterminate him. It turned out to be not so easy as that, though. It seems that in newer firefox installations, extensions just put an encoded .xpi file in your extensions directory. The extension files themselves then seem to be scattered across your hard drive.<br /><br />Since I had recently installed my printer, I looked for the .xpi file with the most recent modification date (you can click on the date column header in windows explorer to sort by date). I moved the file named "{578e7caa-210f-4967-a0d3-88fe5b59a39f}.xpi" out of the extensions directory, restarted firefox, and voila! No more SmartPrintButton extension. Phew.<br /><br />Your SmartPrintButton file might be named something different than mine. And if you didn't install your printer recently, you might then have to take out extension .xpi files in groups, restarting firefox each time, to figure out which file corresponds to SmartPrintButton. But armed with these techniques, you should be able to hunt down the right file.<br /><br />I hope this helps some folks out there!Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-13028095209401400252011-11-06T12:12:00.000-08:002011-11-06T12:14:59.945-08:00Bad Portents: the death of Google Desktop SearchI didn't see any coverage of <a href="http://googledesktop.blogspot.com/2011/09/google-desktop-update.html">the official death of Google Desktop Search (GDS)</a> a couple months ago. While it may have gone quietly into that dark night, its passing troubles me. Back in the XP days, GDS was under active development and it had an absolutely killer set of features, its search was unsurpassed, and it handled all sort of various and sundry types of information.<br /><br />Nothing has come even close to GDS since then (on windows at least - i can't speak for osx). Windows Search blows chunks, and all its alternatives are even worse. I have no idea what Google's calculation was on this one. Certainly they realized that windows search tools suck. Maybe they just didn't see profit in it (not like that's stopped them in the past)?<br /><br />It's been a frustrating couple years as google has let GDS fester, has let its desperate user base senesce with a sense of bitterness, and finally called the time of death as September 2011. This funeral leaves me with no closure. I need desktop search. You need desktop search. We all need desktop search. Who's going to give it to us? Certainly not Microsoft. If not google, then who?Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com3tag:blogger.com,1999:blog-5422202181497793480.post-41578163746708237502011-09-27T08:04:00.000-07:002011-09-27T08:16:27.783-07:00What is really needed for web privacyI've blogged in the past about various tools that help keep your web-wanderings to yourself. Facebook has recently come under some scrutiny for not erasing some tracking cookies (https://nikcub.appspot.com/logging-out-of-facebook-is-not-enough), and while it's no surprise, it does remind us that the big brothers of the world are watching!<br /><br />What we should be aware of is that, while such cookies make it more convenient for facebook to track you, they can do it without them! Browser fingerprinting is amazingly effect (see http://panopticlick.eff.org/), and when intelligently tied together with IP-address information, can give the big brothers an excellent idea of who it is that's pinging them and from where. That is to say, they probably know, to a disturbingly-large extent, where you've been online. If they haven't actually analyzed their logs to figure that out, they could! (Though the browser fingerprinting would require that they have more than just a typical web server log of your traffic).<br /><br />CSFire and Adblock Plus with customized filters can do a great job of keeping our wanderings to ourselves (see http://albosure.blogspot.com/2011/02/block-facebook-ads-on-other-websites.html and http://albosure.blogspot.com/2010/04/plugging-privacy-leaks-with-csfire.html). But what is really needed in order to be invisible to the big brothers, but to still be able to see their content, is a selective-proxy tool that will suck down facebook, google, etc, content via proxies while on third-party sites, but plugs you in directly when you're actually browsing their sites. As it stands now, privacy requires a level of geekiness and paranoia such that only a few tenths of a percentage point of folks out there (probably even less than that) will actually go through the trouble of implementing what's necessary. We need a plugin (and some laws too). Anyone out there feel like developing it?Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com1tag:blogger.com,1999:blog-5422202181497793480.post-57253884936534777092011-02-24T06:58:00.000-08:002011-02-24T07:35:03.737-08:00Facebook is following you - here's how to stop themI've written before about how to set up firefox to increase your privacy and security, and to keep corporate big brother from tracking your actions around the web. Now I want to step that up just a little bit.<br /><br />With the previous setups I've mentioned, even if you see a facebook ad on some other website, facebook will have a difficult time knowing it's you, and knowing where the request came from. For example, let's say I visit lifehacker.com and I see a facebook ad. If I don't implement any security/privacy tools, I will send a cookie to facebook, and so facebook will know that I visit lifehacker (and which specific article I'm reading, and how often I visit, and at what times, etc - kinda scary, eh?). If, however, I implement the security precautions I lay out in my previous posts, I won't send facebook any cookies when I see their ad on lifehacker, and I won't send them a referrer string, so this makes it difficult for facebook to know just who it is that is viewing their ad.<br /><br />Facebook still has at least one piece of information with which they could identify me - my IP address. (There are yet other ways they can still ID us: see <a href="https://panopticlick.eff.org/">EFF's Panopticlick</a> and <a href="https://panopticlick.eff.org/browser-uniqueness.pdf">their paper</a> for some awesome work on the subject). That is, as long as I'm viewing a facebook ad (and not using VPN or some proxy service such as TOR), facebook knows the IP address of the computer viewing their ad. If I'm behind a corporate firewall, with lots of other people viewing facebook and facebook ads, it will still be difficult for facebook to know who I am. If, however, I'm browsing from home or some other place where there aren't many folks viewing facebook with the same IP address, facebook has a much better chance of knowing that I'm viewing their ad, and perhaps from which site, even if I have all the other previously-mentioned privacy tools installed.<br /><br />You might be asking yourself, "ok, so is there ANY way to be sure that facebook doesn't follow me around the web and build a more complete profile of me than perhaps even my best friends or spouse have?" The answer is yes, of course. If we completely block all requests to facebook when we're not on a facebook page, we'll never see the facebook ad in the first place, and facebook will never get a request from our browser when we're outside of facebook. Hence, they'll never know anything about us, except for what we explicitly tell them and do while on facebook.com itself. Here's how you do it:<br /><br />First, <a href="https://addons.mozilla.org/en-us/firefox/addon/adblock-plus/">install Ad-Block Plus in firefox</a> or <a href="https://chrome.google.com/extensions/detail/cfhdojbkjhnklbpkdaibdccddilifddb">chrome</a> if you haven't already. Go ahead and add the Easylist+Tracking filter while you're at it.<br /><br />Second, click on the ad-block plus icon, then click preferences. Then click "Add Filter...", and add the following four lines, one by one <br />(props to Saudrapsmann and <a href="http://lifehacker.com/#!5542041/block-sites-from-using-your-facebook-login-with-adblock-plus">lifehacker</a>):<br /><br /><blockquote><br />||facebook.com^$domain=~facebook.com|~facebook.net|~fbcdn.com|~fbcdn.net<br /><br />||facebook.net^$domain=~facebook.com|~facebook.net|~fbcdn.com|~fbcdn.net<br /><br />||fbcdn.com^$domain=~facebook.com|~facebook.net|~fbcdn.com|~fbcdn.net<br /><br />||fbcdn.net^$domain=~facebook.com|~facebook.net|~fbcdn.com|~fbcdn.net</blockquote><br /><br />And that's it. For the techy interested, you can read about ad-block plus filter syntax <a href="http://adblockplus.org/en/filters">here</a>. In essence, the domain right after the "||" is saying which domains to block. The "domain=" option specifies which domains this rule should apply do (i.e. only apply this rule when viewing a webpage from these domains). The tilde (~) is a negation argument, and hence this rule gets applied on all pages *except* facebook.com/net/fbcdn.com/net. <br /><br />It wouldn't be a horrible idea to do this for other big-brother companies such as google as well, as long as not seeing their content on other pages won't kill you.<br /><br />In the future, what i'd really like to see are tools to allow such content to be delivered to the browser, but through a TOR-like proxy architecture. That way, the increasingly-interconnected web won't completely break, but our privacy will remain more-or-less intact.Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com4tag:blogger.com,1999:blog-5422202181497793480.post-40644127065643101632010-05-21T13:47:00.001-07:002010-05-21T13:53:41.039-07:00Paste your tracking number directly into google<span class="ctedit">anyone ever send you a tracking number, but didn't tell you which service they sent it with? Or don't feel like bumbling around to find the tracking page for the usps, ups, or whatever?<br /><br />Just paste the tracking number into google directly - the G groks it and shows you a link to the appropriate tracking page automatically! nice.<br /></span>Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com0tag:blogger.com,1999:blog-5422202181497793480.post-46152978963106082562010-05-07T09:20:00.000-07:002010-05-07T11:10:41.445-07:00FDA requests comments for front-of-packaging labelsThe FDA is requesting comments for potentially new front-of-packaging food labels. Their request can be found <a href="http://www.regulations.gov/search/Regs/home.html#documentDetail?R=0900006480ae39f5">here</a>.<br /><br />You can leave a comment for the FDA <a href="http://www.regulations.gov/search/Regs/home.html#submitComment?R=0900006480ae39f5">here</a>.<br /><br />My comment to the FDA:<br /><br />It has become common knowledge that nutrition and health includes not just the contents of the food itself, but the impact of the production of that food on the environment in which we breathe, drink, recreate and live. Therefore, nutritional labeling should not only reference the material contained in package, but it should also indicate the major environmental impacts of producing that food.<br /><br />Such a indication could be implemented graphically, with labels for different ecosystems impacted, in which ways, and the relative magnitude of those impacts. This would include not just the production, but also transportation, marketing and other activities associated with the life-cycle of the food item.<br /><br />When such information is made accessible to the public, citizens will be able to make informed choices on how they impact their environment; such a choice is impossible at the moment without available information.<br /><br />The impacts of the ability for the public to make such choices will be profound, as low-carbon-emission foods, for example, would be incentivized over high-carbon-emission foods. Local food would likely be incentivized, but not in all cases. Overall, environmentally-friendly practices would be incentivized.<br /><br />The hurdle of creating the capacity to determine such impacts would be overcome with an investment on the part of the government into a life-cycle analysis software system which would be available for all producers to use.Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com1tag:blogger.com,1999:blog-5422202181497793480.post-47807812852269175832010-04-27T06:18:00.000-07:002010-05-27T10:07:01.941-07:00Plugging privacy leaks with CsFireIn my <a href="http://albosure.blogspot.com/2010/04/evading-big-brother-online.html">last post</a>, I detailed a number of firefox add-ons which serve to keep your web activities as private as possible without going through the hassle of using anonymizers. I mentioned that a major hole in the assembly of tools I outlined there was the lack of 3rd-party cookie blockers. That means that, as you browse to example.com, for example, google can track you there if example.com has links to google embedded in its pages. Ad-block plus doesn't help us there because google isn't identified as an advertiser - only its Adsense servers are. This goes for facebook, nytimes, and any other sites not specifically targeted as advertisers, and for whom we have allowed cookies to operate.<br /><br />This is the new, real threat to privacy - corporations whose services we would like to use to some extent, but whose prying eyes know no limits: I couldn't get by without google these days, but I certainly don't want them tracking me all over the place.<br /><br />The only answer I've found so far for this issue is Philippe De Ryck's add-on called <span style="font-weight: bold;">CsFire</span> [<a href="https://addons.mozilla.org/en-US/firefox/addon/58189">addons.mozilla.org</a>]. CsFire blocks 3rd-party cookies, and more. It's pretty complex, but it will work out of the box, without modification, for most users (though it allows requests to google and facebook by default - you can change that by implementing "local" policies) . With CsFire, you can: block 3rd-party cookies; block 3rd-party HTTP authentication (with the newest versions of firefox); block 3rd-party content from loading; customize behavior on a origin-destination site pair basis; and more.<br /><br />If you're concerned about corporate invasions of your privacy like I am, I highly recommend using this add-on - it goes a long way to protecting what is being put at risk by ubiquitous services such as google, and social networks such as facebook. Unless you configure it to completely block 3rd party content (I'm considering doing this for facebook and google), these corporations may still be able to track you to an extent via your IP address, but it will be hard for them if you're behind a NAT with lots of other computers. Even if you're not behind a NAT, this add-on will still complicate their tracking tasks *greatly*, and will go a long way towards protecting your privacy.Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com2tag:blogger.com,1999:blog-5422202181497793480.post-82470559452488028562010-04-10T09:44:00.000-07:002010-04-10T10:07:17.471-07:00Evading big brother onlineMore than security from hackers while browsing the web, I am concerned about being tracked by big brother corporations. These guys put little tracking bugs in various partner websites, and on their own, to keep track of what you’re doing, and where. I’m sharing my setup for firefox below in the hopes that y’all will benefit from my experience, and will help keep privacy a reality in the 21st century. This setup will incidentally help you with hacker security as well (though the main add-on for that is Noscript, mentioned below).<br /><br />The main rule in setting these applications up is that you want a “Global Deny” policy, with a user-defined “whitelist” of trusted sites. The whitelist gets built up as you move around the web and slowly add sites to your list. That is, you don’t have to spend time creating these lists before you start, or before you’ll be secure. The point is that it’s easy. Having said that, this setup probably ain’t for granny. But if you have just a modicum of savvy, or the desire, this should work for you just fine.<br /><br />Here are the add-ons I’ve installed in Firefox (and sometimes my email client, Thunderbird, too):<br /><br /><a href="https://addons.mozilla.org/en-US/firefox/addon/1865"><span style="font-weight: bold;">Ad-Block Plus</span></a><br /><br />When you install this, you can also install a number of “filters”. I use <span style="font-weight: bold;">EasyList, EasyPrivacy </span>and <span style="font-weight: bold;">fanboy-adblock</span>. Don’t install a bunch of filters – it will be redundant, and will slow down your browser.<br /><br />This add-on is not only good to block tracking sites, but it will block advertisements as well. I didn’t care about the latter initially, but there wasn’t a good way to just block tracking sites, so I installed the full set of filters. And boy – how much nicer the web became! Not only are ads generally annoying and intrusive (and we already see enough of them every day), they take up your bandwidth! For the bandwidth-limited, this add-on will improve your web experience even further by speeding things up.<br /><br /><a href="https://addons.mozilla.org/firefox/addon/5207"><span style="font-weight: bold;">CS Lite</span></a><br /><br />This add-on controls what sites can put cookies on your browser.<br />You should set the preferences to “Deny cookies globally”. Whenever you come across a site that you trust, and that doesn’t seem to be working for you, click on the CS Lite icon in the status bar and set the site to either “allow cookies”, “allow cookies for session”, or “temporarily allow cookies”. I set up most of my cookies for the session (which deletes them after a time, so you have to log in again later).<br /><br /><span style="font-weight: bold;"><a href="https://addons.mozilla.org/en-US/firefox/addon/953">Refcontrol</a></span><br /><br />This keeps your browser from telling sites where you came from. This is important because you can be tracked by these referrer strings. Let’s say you’re on your account page for a site, and there’s an ad from doubleclick on that page. Normally, even if you’re blocked cookies from doubleclick, they will see the site you came from when you see their advertisement. If there is identifiable information in the web address where you are (for example, an account number embedded in the URL), doubleclick could track you even without cookies (theoretically). Regardless, in general, it’s not a great idea to be sending around such information to sites that log your visits.<br /><br />If you have problems with a site, right-click on the ref-control icon in the status bar, go to options, and add the site to your exceptions list (just like the cookies).<br /><br />My settings:<br />Default set to <block><block>Block<br /><br /><a href="https://addons.mozilla.org/en-US/firefox/addon/6623"><span style="font-weight: bold;">Betterprivacy</span></a><br /><br />Flash can set cookies also, but they’re different from normal cookies, so you need a different add-on to control them.<br /><br />My settings:<br />Set to delete cookies without asking upon shutdown<br /><br /><br />Other interesting plugins:<br /><br /><a href="https://addons.mozilla.org/addon/722"><span style="font-weight: bold;">Noscript</span></a> – starting to use this myself. Again, deny globally, with a user-defined whitelist.<br /><br /><a href="https://addons.mozilla.org/en-US/firefox/addon/9609"><span style="font-weight: bold;">Ghostery</span></a> – show web bugs and other thingys that ad-block doesn’t catch.<br /><br /><a href="http://mrl.nyu.edu/%7Edhowe/trackmenot/"><span style="font-weight: bold;">Trackmenot</span></a> – sends random searches to search engines in order to muddy their characterization of your web behavior.<br /><br /><br />A Note:<br /><br />One problem with this setup is that I need to allow some sites to track me, such as Google, in order to take advantage of their functionality. Google is the ultimate big brother. What we can do in this case, however, is to block “3rd party cookies”.<br /><br />What that means is that, if you go see the google advertisements to the right of this blog post, google knows that *you* are browsing my blog (if you have google cookies). But those cookies don’t belong to blogspot.com – they belong to google.com. Hence, they are a “3rd party”. By blocking these 3rd party cookies, google and other sites you’ve allowed cookies for will only be able to track you, for the most part, when you’re on their respective pages (such as google.com), but not when you’re on other pages (such as nytimes.com). Unfortunately, CS Lite doesn’t have an option to block 3rd party cookies yet (I submitted a query about it – we’ll see if it gets addressed).<br /><br />Regardless, even if we manage to block 3rd party cookies at some point, Google will still know the IP address from whence the request is coming. So if they’re smart (and they’re pretty smart), they’ll still probably be able to track you with your IP if you’ve visited google.com recently (which you probably have). One answer to the IP-tracking problem is to block ad in the first place, and that’s where Ad-block Plus comes in. By blocking your browser from getting the ads from google’s (or others’) ad-servers in the first place, google will have no way of knowing you’ve gone to pages where their ads are shown. Another option is to use a TOR (http://www.torproject.org/) proxy setup which changes your apparent IP address every so often, but that’s getting a bit beyond the scope of things here. Feel free to look it up online.<br /><br /></block></block>Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com14tag:blogger.com,1999:blog-5422202181497793480.post-60256320006653233922008-12-16T14:13:00.000-08:002008-12-16T14:14:06.834-08:00worse yet or worse still?Which phrasing is correct: "worse yet" or "worse still"? Perhaps both are?Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com2tag:blogger.com,1999:blog-5422202181497793480.post-73502440798786961552008-10-28T08:14:00.001-07:002008-10-28T09:04:40.077-07:00Faxing from your computer for almost freeWelcome to the first post of Albo Wanderings. It will be hosted here on blogger until my server gets access to the web again in Florida. I expect that, by reading this blog, you'll get a good sense of all my quirks - my penny pinching/efficient tendencies, my random interests, electronics, ecology, computers, good deals, hacking your life, ....<br /><br />To start things off, I thought I'd let you all in on my little faxing setup I have going. I'm running a risk here that my faxing paradise will be discovered and some exec will tweak their service agreement just to spite me. Oh well, I can't resist.<br /><br />I was searching around for fax services that served the little guy - someone who sends and receives faxes once every couple months. For such low usage, having a fax machine is unwieldy, and especially so for someone who travels around as much as I do. An online solution is clearly superior. The problem is that most services charge you a monthly fee. For a small guy, that would probably add up to about $10 / fax.<br /><br />The one service I found that doesn't charge you a monthly fee is Interfax (http://www.interfax.net/). After you set up your account, you load it up with $10 or so and you pay $0.11 per page to fax. Most importantly, as far as I can tell, you can send one fax every 10 years, and they don't penalize you for it. To send faxes via this service, you email an address at fax.tc. For example, if you wanted to send a fax to information, you would email "+18005551212@fax.tc". You can add all sorts of attachments, and the most important part to remember is that, if you also want the body of your email included as a page as well, you write "/b" at the end of the subject line. It's probably not a great solution for grandma, but if you're have even the smallest inclination towards command lines, this works great. (Interfax seems to actually focus on webservices, and providing their service to other websites and tools via that route, but they've nicely kept open an email interface as well).<br /><br />Interfax is only an outbound solution, though. They charge a monthly fee of about $13 for receiving faxes. So how do we get around that? You can actually sign up for an inbound-only account at eFax (http://www.efax.com), for free! That's right - they'll give you a number, in a Salt Lake City area code or something like that, and when you receive a fax, they send you an email. You need to install the eFax messenger software, but it's no big deal, and there's no spyware included that I know of. eFax insists on sending you occassional promotional emails, which is basically the price you pay for their free service. For someone who is pretty sensitive to spam, it's still worth it. They only send out 2-3 emails a month or so, if memory serves.<br /><br />So there you go. Tell 'em I sent you. Happy faxing - what a great retro technology that just doesn't seem to be going away. And if interfax or efax is reading this: please don't change a thing!Anonymoushttp://www.blogger.com/profile/04412699314995180458noreply@blogger.com3