Tales of Wrangling Geodata2022-11-21T01:57:55+00:00http://geodatawrangler.azym8.comPaul McCombsDown the Emacs Rabbit Hole2019-11-20T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2019/11/20/Rabit-Hole<p>I update this post periodicaly in order to keep my Emacs configuration notes all together. The main content follows the change log.</p>
<h2 id="change-log">Change Log:</h2>
<ul>
<li><strong>November 20, 2019</strong> Replace linum mode with display-line-numbers mode</li>
<li><strong>May 17, 2019</strong> Change the theme and the default font</li>
<li><strong>April 7, 2019</strong> Auto Complete Config fix; Add Saving Sessions and CSV-Mode to list of enhancements to explore</li>
<li><strong>March 7, 2019</strong> Added list of enhancements to explore, and added a note about Orgzly.</li>
<li><strong>January 17, 2019</strong> Figured out how to push-to-deploy.</li>
<li><strong>December 1, 2018</strong> Fixed formatting and updated with experience installing Emacs on a completely fresh 2018 Mac Mini with Mojave.</li>
<li><strong>October 4, 2018</strong> Initial blog posted.</li>
</ul>
<h2 id="im-late">I’m Late…</h2>
<p>Over the years I have followed the <a href="https://www.gnu.org/fun/jokes/gospel.html">holy</a> <a href="https://en.wikipedia.org/wiki/Editor_war">war</a> between <a href="https://en.wikipedia.org/wiki/Vi">vi</a> and <a href="https://en.wikipedia.org/wiki/Emacs">Emacs</a> with some interest. As a university student and a GIS Analyst using various versions of Unix, I was handed vi and taught to use it. vi was omnipresent and could be learned to a passable level in a short amount of time. So, I was a vi user. Emacs was alluring though. The old joke, that Emacs is “a great operating system, lacking only a decent editor”<a href="#foot1"><sup>1</sup></a>, was both a draw to and a caution against it’s use. I tried it out a few times, but the requirement to learn so many commands with multiple meta keys was too much bother. I would give up after an hour or two, when I had to look up open a file for the eighth time.</p>
<p>At the same time, I was experimenting with various ways of keeping track of my huge backlog of “to do” items. Often hundred’s of items long. Tracking it in a Word document, tracking it in an excel spreadsheet. Using tabs in steno note book. When I finally gave up the fight and switched from paper notes to Microsoft OneNote, I started keeping the list in there. It did not work that well. Also, I have an ever growing fear that when Microsoft decides to move from OneNote to the next great thing, all my notes will be locked inside its proprietary file format. I’ve been burned with that problem in the past.</p>
<p>I became aware of Org Mode. It was built within the platform that is Emacs. It uses plain text files. It has fantastic miraculous sounding capabilities. It can be version controlled with git. I can store it in the cloud. I’ve become familiar with using Markdown and Jekyll to publish my blogs on GitHub and I love the plain text files being used for markup. So I took a serious stab at using Emacs for the first time.</p>
<p>My initial attempt is partially documented below in <strong><em><a href="#aquamacs">Aquamacs</a></em></strong>. While at the same time using it on my Windows 8.1 computer at work. That became frustrating because of the differences associated with Aquamacs. So I’ve set out to document fully how I got Emacs with Org Mode and all its other wonders in <strong><em><a href="#homebrew">Homebrew</a></em></strong>.</p>
<h2 id="homebrew">Homebrew</h2>
<ol>
<li>Install Emacs
<ol>
<li>Install <a href="https://brew.sh/">Homebrew</a></li>
<li>At terminal prompt type <code class="language-plaintext highlighter-rouge">brew cask install emacs</code></li>
<li>For MacOS 10.14 (Mojave) it is necessary to allow Emacs to control your computer.
It will prompt you when it tries to do something that is not yet permitted. This is a setting in the <em>Security & Privacy</em> panel in <em>System Preferences</em> under the <em>Privacy</em> tab with <em>Accessibility</em> selected.</li>
</ol>
</li>
<li>Configure Interface - add items to init.el
<ol>
<li>Line Numbers:
<ul>
<li>create file <code class="language-plaintext highlighter-rouge">~/.emacs.d/init.el</code> and add:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(global-display-line-numbers-mode t)
</code></pre></div> </div>
</li>
<li><strong>Note</strong>: Previously I was using linum-mode, which predates <a href="https://www.emacswiki.org/emacs/LineNumbers#toc1">display-line-numbers-mode</a>. I was experiencing significant typing delay in an approximately 2,000 line org file on Windows 10 at work. I determined that turning off linum removed the delay. I have changed my MacOS configuration as well for consistency.</li>
</ul>
</li>
<li>Recent Files
<ul>
<li>Add to init.el:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(recentf-mode 1)
(setq recentf-max-menu-items 25)
(global-set-key "\C-x\ \C-r" 'recentf-open-files)
</code></pre></div> </div>
</li>
</ul>
</li>
<li>Visual Line Mode - wraps lines on word breaks
<ul>
<li>Add to init.el: <code class="language-plaintext highlighter-rouge">(global-visual-line-mode t)</code></li>
</ul>
</li>
<li><a href="https://www.gnu.org/software/emacs/manual/html_node/efaq/Matching-parentheses.html">Turn on Parenthesis Matching</a>
<ul>
<li>Add <code class="language-plaintext highlighter-rouge">(show-paren-mode 1)</code> to init.el</li>
</ul>
</li>
<li>Configure MELPA repository for packages
<ul>
<li>Add to init.el
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(require 'package)
(add-to-list 'package-archives
'("melpa-stable" . "https://stable.melpa.org/packages/"))
(package-initialize)
</code></pre></div> </div>
</li>
</ul>
</li>
</ol>
</li>
<li>Markdown Mode for Emacs
<ol>
<li><a href="https://jblevins.org/projects/markdown-mode/">Install package</a>.
<ul>
<li>M-x package-install [RET] markdown-mode [RET]</li>
</ul>
</li>
<li><a href="https://pandoc.org/">Pandoc</a>
<ul>
<li>At Terminal Prompt type <code class="language-plaintext highlighter-rouge">brew install Pandoc</code></li>
</ul>
</li>
<li>Add to init.el inside the <code class="language-plaintext highlighter-rouge">(custom-set-variables</code> section before the final <code class="language-plaintext highlighter-rouge">)</code>:
<ul>
<li><code class="language-plaintext highlighter-rouge">'(markdown-command "/usr/local/bin/pandoc")</code></li>
</ul>
</li>
</ol>
</li>
<li><a href="https://orgmode.org/worg/org-tutorials/orgtutorial_dto.html">Org Mode</a>
<ol>
<li>Add to init.el:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(require 'org)
(define-key global-map "\C-cl" 'org-store-link)
(define-key global-map "\C-ca" 'org-agenda)
(setq org-log-done 'time)
</code></pre></div> </div>
</li>
<li>
<p>For each .org file you want to participate in the Agenda open the file and issue command C-c [</p>
</li>
<li>Create a <a href="https://orgmode.org/worg/org-tutorials/org-custom-agenda-commands.html">Custom Agenda Command</a>
<ul>
<li><strong>NOTE</strong> you can use: M-x customize-variable [RET] org-agenda-custom-commands</li>
<li>Otherwise, add the following text to init.el inside the <code class="language-plaintext highlighter-rouge">(custom-set-variables</code> section. The “n” option was included already I added the “L” section to do a <a href="https://emacs.stackexchange.com/questions/12517/how-do-i-make-the-timespan-shown-by-org-agenda-start-yesterday">2 week agenda</a> with log mode turned on.
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> '(org-agenda-custom-commands
(quote
(("L" "Time (L)ine - Biweekly Report" agenda ""
(
(org-agenda-start-day "-14d")
(org-agenda-span 14)
(org-agenda-start-on-weekday nil)
(org-agenda-start-with-log-mode '(closed clock state))
)
)
("n" "Agenda and all TODOs"
((agenda "" nil)
(alltodo "" nil))\
nil))))
</code></pre></div> </div>
</li>
</ul>
</li>
<li>Org files in the cloud. I have started experimenting with <a href="http://www.orgzly.com">Orgzly</a> and keeping an org file synced on my Android phone, my laptop, and my desktop. It’s working OK so far.</li>
</ol>
</li>
<li>Auto-Complete
<ol>
<li><a href="https://github.com/auto-complete/auto-complete">Install package</a>.
<ul>
<li>M-x package-install [RET] auto-complete [RET]</li>
</ul>
</li>
<li><a href="https://emacs.stackexchange.com/questions/18982/how-do-i-make-auto-complete-enabled-by-default">Add to init.el</a>:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>;; Set up configuration for auto complete
;; Among other things the first line pops-up doc strings for functions
;; highlighted in AC's suggestions list, while in Emacs-Lisp mode.
(ac-config-default)
(global-auto-complete-mode t)
(add-to-list 'ac-modes 'markdown-mode)
(add-to-list 'ac-modes 'org-mode)
</code></pre></div> </div>
</li>
</ol>
</li>
<li><a href="https://stackoverflow.com/questions/19022015/emacs-on-mac-os-x-how-to-get-spell-check-to-work">Spell Checking</a>
<ol>
<li>At Terminal Prompt
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew install aspell --with-all-langs
which aspell
</code></pre></div> </div>
</li>
<li>Copy the response from last command. Similar to: <code class="language-plaintext highlighter-rouge">/usr/local/bin/aspell</code></li>
<li>Run Customize… from Edit->Spell menu</li>
<li>Set ispell-program-name variable to <code class="language-plaintext highlighter-rouge">/usr/local/bin/aspell</code> or other return from earlier prompt command.</li>
<li>Run Spell-Check Buffer from Edit->Spell menu</li>
<li><strong>NOTE</strong> Flyspell can work but needs a functional mouse-2 click which requires hackery for macOS track-pad.</li>
</ol>
</li>
<li><a href="https://magit.vc/manual/magit/Installing-from-an-Elpa-Archive.html#Installing-from-an-Elpa-Archive">Magit</a>: git version control
<ol>
<li>M-x package-refresh-contents [RET]</li>
<li>M-x package-install [RET] magit [RET]</li>
<li>Test version numbers: M-x magit-version [RET]
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Magit 2.90.1, Git 2.17.2 (Apple Git-113), Emacs 26.1, darwin
</code></pre></div> </div>
</li>
<li>Add a global key binding for magit-status to init.el
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>;; Configure magit
;; https://magit.vc/manual/magit/Getting-Started.html#Getting-Started
(global-set-key (kbd "C-x g") 'magit-status)
</code></pre></div> </div>
</li>
<li><strong>NOTE</strong> <a href="https://codingitwrong.com/wiki/git-push-to-deploy">Regarding push-to-deploy</a>
One of the things I hoped to accomplish with Magit in emacs is a simple solution for publishing my static hand-crafted <a href="https://www.lazym8.com">personal website</a>. I was struggling because default behavior requires using two branches and then automating the merge to the master branch using <a href="https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks">git hooks</a>, which was on my list to figure out, but since I don’t ever edit the files on the server, push-to-deploy is more straight-forward. In the remote repository on the web server you only need to configure git with <code class="language-plaintext highlighter-rouge">git config receive.denyCurrentBranch updateInstead</code></li>
<li><strong>NOTE</strong> I have not yet learned how to clone a repository with Magit</li>
</ol>
</li>
<li>Time to tryout a different look: Change Default Font and Theme
I elected to use the handy GUI interface for this as I did it originally on my work computer (Windows) quickly while on break.
<ol>
<li>Download the hipster coding font of your choice. I ended up downloading Fira Mono and Ubuntu Mono from <a href="https://fonts.google.com/?category=Monospace">Google Fonts</a>. Install them in the usual fashion for your <a href="https://support.apple.com/en-us/HT201749">operating</a> <a href="https://answers.microsoft.com/en-us/windows/forum/windows_10-start/installing-new-fonts-on-windows-10/09e5d7c3-0870-4998-adb9-fc5391e0fb49">system</a>.</li>
<li>Click on <em>Set Default Font…</em> on the <em>Options</em> menu and find the font you chose on the list.</li>
<li>Click on <em>Custom Themes</em> on the <em>Customize Emacs</em> sub-menu on the <em>Options</em> menu and browse the themes listed there. I selected <em>leuven</em> because of how it renders an org mode document. After you decide which one to keep click the <em>Save Theme Settings</em> button. <strong>Just kidding!</strong> That worked on my Windows 10 Computer at work, but not on my Mac. I found <a href="https://stackoverflow.com/questions/1257426/emacs-mac-osx-and-changing-default-font">some directions on Stack Overflow</a>, which were slightly dated.<br />
<img src="/images/leuven_org_mode.png" alt="org mode with leuven theme" /></li>
<li>These Directions worked on MacOS for me:
<ol>
<li>Type: M-x customize-face [RET]</li>
<li>At “Customize face (…):” prompt type: default [RET]</li>
<li>Edit <em>Font Family</em> value to match what you see in the <em>Fonts</em> dialog. In my case “Fira Mono”.<br />
<img src="/images/fonts.png" alt="_Fonts_ Dialog" /></li>
<li>Uncheck all of the other options. If you do not those settings will override the settings in any subsequent theme that you choose in the <em>Custom Themes</em> buffer.</li>
<li>Click the _Apply and Save__ button. This will make a change in your init.el file like this:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(custom-set-faces
;; custom-set-faces was added by Custom.
;; If you edit it by hand, you could mess it up, so be careful.
;; Your init file should contain only one such instance.
;; If there is more than one, they won't work right.
'(default ((t (:family "Fira Mono")))))
</code></pre></div> </div>
</li>
<li>Exit the buffer using the graphic button in he tool bar at the top of the screen.</li>
</ol>
</li>
</ol>
</li>
<li>Topics to explore in the future:
<ol>
<li>
<p><a href="https://www.emacswiki.org/emacs/CsvMode">CSV-Mode</a> - Allows column alignment and sorting by column. Also transposing columns and rows.</p>
</li>
<li>
<p><a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/Saving-Emacs-Sessions.html">Saving Sessions</a> This looks very helpful to get figured out, so I don’t have to keep loading the same files over and configuring my windows.</p>
</li>
<li>
<p><a href="https://www.gnu.org/software/emacs/manual/html_node/gnus/index.html">GNUS</a> - research for use as e-mail client. Rejected Mu4e as too complicated.</p>
</li>
<li>
<p><a href="https://github.com/skeeto/elfeed">Elfeed</a> - RSS Feed Reader</p>
</li>
<li>
<p><a href="https://www.emacswiki.org/emacs/BookmarkPlus#toc60">Bookmark Plus</a></p>
</li>
<li>
<p><a href="https://www.emacswiki.org/emacs/BufferMenu">Buffer Menu</a></p>
</li>
<li>
<p><a href="https://www.emacswiki.org/emacs/AbbrevMode">Abbrev Mode</a></p>
</li>
</ol>
</li>
</ol>
<h2 id="aquamacs">Aquamacs</h2>
<h3 id="things-installed-or-configured">Things installed or configured</h3>
<ul>
<li>line numbers</li>
<li>word wrap</li>
<li>auto-complete</li>
<li>org-mode</li>
<li>markdown-mode</li>
<li>pandoc</li>
</ul>
<h3 id="config-customization">Config customization</h3>
<p><em>/Users/paulmccombs/Library/Preferences/Aquamacs Emacs/Preferences.el</em></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>;; This is the Aquamacs Preferences file.
;; Add Emacs-Lisp code here that should be executed whenever
;; you start Aquamacs Emacs. If errors occur, Aquamacs will stop
;; evaluating this file and print errors in the *Messages* buffer.
;; Use this file in place of ~/.emacs (which is loaded as well.)
(require 'org)
(define-key global-map "\C-cl" 'org-store-link)
(define-key global-map "\C-ca" 'org-agenda)
(setq org-log-done t)
(setq org-agenda-files (list "~/org/work_org_mode/work.org"))
</code></pre></div></div>
<p><em>/Users/paulmccombs/Library/Preferences/Aquamacs Emacs/customizations.el</em></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>;; Added by Package.el. This must come before configurations of
;; installed packages. Don't delete this line. If you don't want it,
;; just comment it out by adding a semicolon to the start of the line.
;; You may delete these explanatory comments.
(package-initialize)
(custom-set-variables
;; custom-set-variables was added by Custom.
;; If you edit it by hand, you could mess it up, so be careful.
;; Your init file should contain only one such instance.
;; If there is more than one, they won't work right.
'(ac-modes
(quote
(emacs-lisp-mode lisp-mode lisp-interaction-mode slime-repl-mode nim-mode c-mode cc-mode c++-mode objc-mode swift-mode go-mode java-mode malabar-mode clojure-mode clojurescript-mode scala-mode scheme-mode ocaml-mode tuareg-mode coq-mode haskell-mode agda-mode agda2-mode perl-mode cperl-mode python-mode ruby-mode lua-mode tcl-mode ecmascript-mode javascript-mode js-mode js-jsx-mode js2-mode js2-jsx-mode coffee-mode php-mode css-mode scss-mode less-css-mode elixir-mode makefile-mode sh-mode fortran-mode f90-mode ada-mode xml-mode sgml-mode web-mode ts-mode sclang-mode verilog-mode qml-mode apples-mode markdown-mode gfm-mode org-mode)))
'(aquamacs-additional-fontsets nil t)
'(aquamacs-customization-version-id 308 t)
'(aquamacs-tool-bar-user-customization nil t)
'(global-auto-complete-mode t)
'(global-linum-mode t)
'(global-visual-line-mode t)
'(markdown-command "pandoc")
'(ns-tool-bar-display-mode (quote both) t)
'(ns-tool-bar-size-mode (quote regular) t)
'(package-selected-packages (quote (auto-complete auto-correct markdown-mode)))
'(text-mode-hook nil)
'(visual-line-mode nil t))
(custom-set-faces
;; custom-set-faces was added by Custom.
;; If you edit it by hand, you could mess it up, so be careful.
;; Your init file should contain only one such instance.
;; If there is more than one, they won't work right.
)
</code></pre></div></div>
<h2 id="foot-notes">Foot Notes</h2>
<p><a name="foot1"></a>[1] <a href="https://en.wikipedia.org/w/index.php?title=Editor_war&oldid=862368474">Editor war, <em>Wikipedia, The Free Encyclopedia</em></a> (last visited Oct. 11, 2018).</p>
QGIS: A Free and Open Source Geographic Information System2018-06-06T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2018/06/06/GIS-and-You-QGIS<p>I have published an article <a href="https://gisandyou.org/2018/05/21/a-free-and-open-source-geographic-information-system/">introducing QGIS to an audience familiar with commercial GIS software</a> on the blog for <a href="https://www.kingcounty.gov/services/gis.aspx">King County GIS</a>: <a href="https://gisandyou.org/">GIS & You</a>.</p>
A Government Worker at SOTMUS 20172018-02-01T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2018/02/01/GIS-and-You<p>I’ve got a new article about my <a href="https://gisandyou.org/2018/01/31/public-data-meets-open-data/">trip to State of the Map US 2017</a> up on the blog for <a href="https://www.kingcounty.gov/services/gis.aspx">King County GIS</a>: <a href="https://gisandyou.org/">GIS & You</a>.</p>
<p>It’s focused on how municipal government can benefit from and contribute to <a href="https://www.openstreetmap.org">OpenStreetMap</a>.</p>
Dillinger.io - A Quick Way to Edit Markdown2017-02-17T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2017/02/17/QGIS-Dillinger<p>I got a tip from a friend today to try writing my Markdown with <a href="http://dillinger.io/">Dillinger</a>. I don’t think it’s new or anything. When my copy editor complained about the Markdown codes getting in the way of editing. I had her try it, and it solved the problem nicely. I good first experience.</p>
MySQL: What in the Hell is the --secure-file-priv option?2017-02-16T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2017/02/16/secure-file-priv<h3 id="task-retrieve-data-from-a-dump-file">Task: Retrieve Data From a Dump File</h3>
<p>A coworker had some <a href="https://dev.mysql.com/doc/refman/5.7/en/mysqldump-sql-format.html">MySQL dump files</a> but no longer had a MySQL installation. I thought it would be a quick job to convert the dump files into something more portable.</p>
<p>I could do a quick install of MySQL on my Mac, bring in the dump file and export it to a portable format. I’d just use <a href="https://brew.sh/">Homebrew</a> to install it and I’d be rolling. Wait! Homebrew is hosed on my computer. So first a detour to get Homebrew working.</p>
<h3 id="installing-mysql-on-a-mac-with-homebrew">Installing MySQL on a Mac with Homebrew</h3>
<p>The directory permissions were changed when I upgraded to OSX 10.11 (El Capitan) I hadn’t had cause to figure out the <a href="https://digitizor.com/fix-homebrew-permissions-osx-el-capitan/">fix</a>. In the process, I also upgraded to macOS 10.12 (Sierra).</p>
<p>When Homebrew is working it’s a dream. Open source software is installed just like it would be in Linux with <a href="https://wiki.debian.org/Apt">apt</a> or <a href="https://en.wikipedia.org/wiki/Yellowdog_Updater,_Modified">yum</a>. After a bit of googling I found <a href="http://superuser.com/questions/203707/how-to-uninstall-homebrew-mac-os-x-package-manager">instructions to uninstall Homebrew</a>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ruby <span class="nt">-e</span> <span class="s2">"</span><span class="si">$(</span>curl <span class="nt">-fsSL</span> https://raw.githubusercontent.com/Homebrew/install/master/uninstall<span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>Next I followed the <a href="https://brew.sh/">one-step instructions at brew.sh</a></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/usr/bin/ruby <span class="nt">-e</span> <span class="s2">"</span><span class="si">$(</span>curl <span class="nt">-fsSL</span> https://raw.githubusercontent.com/Homebrew/install/master/install<span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>and I was back in Homebrew action.</p>
<p>A not-to-quick</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew install mysql
mysql.server start
</code></pre></div></div>
<p>and I was ready to roll<sup><a href="#myfootnote1">1</a></sup> with MySQL. Or was I?</p>
<h3 id="what-in-the-hell-is-the-secure-file-priv-option">What in the Hell is the –secure-file-priv option?</h3>
<p>With many thanks to <a href="http://stackoverflow.com/questions/17666249/how-to-import-an-sql-file-using-the-command-line-in-mysql">Stack Overflow</a> The dump files were loaded into my brand new MySQL server:
In mysql:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">create</span> <span class="k">database</span> <span class="n">sailwx_info_shiptracks</span><span class="p">;</span>
</code></pre></div></div>
<p>At terminal prompt:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mysql <span class="nt">-u</span> root <span class="nt">-h</span> localhost sailwx_info_shiptracks < duwamish1.sql
mysql <span class="nt">-u</span> root <span class="nt">-h</span> localhost sailwx_info_shiptracks < duwamish2.sql
mysql <span class="nt">-u</span> root <span class="nt">-h</span> localhost sailwx_info_shiptracks < duwamish3.sql
</code></pre></div></div>
<p>After some poking around I found this <a href="https://dev.mysql.com/doc/refman/5.7/en/select-into.html"><code class="language-plaintext highlighter-rouge">SELECT ... INTO OUTFILE</code></a> syntax for MySQL. The more portable alternative to the dump files I’d started out with. I thought my job was complete, but then found an error I didn’t understand.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">select</span> <span class="o">*</span> <span class="k">from</span> <span class="n">reportSources</span> <span class="k">into</span> <span class="n">outfile</span> <span class="s1">'/Users/paulmccombs/reportSources.txt'</span><span class="p">;</span>
<span class="n">ERROR</span> <span class="mi">1290</span> <span class="p">(</span><span class="n">HY000</span><span class="p">):</span> <span class="n">The</span> <span class="n">MySQL</span> <span class="n">server</span> <span class="k">is</span> <span class="n">running</span> <span class="k">with</span> <span class="n">the</span> <span class="c1">--secure-file-priv option so it cannot execute this statement</span>
</code></pre></div></div>
<h3 id="oh-thats-what-it-means">Oh, That’s What it Means!</h3>
<p>I googled my heart out and read many pieces of advice across several different forum postings. No one had a complete answer though, hence I’ve been inspired to share my struggle and triumphant solution. The first key to removing confusion was learning this <a href="https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-6.html">a change at MySQL version 5.7.6</a></p>
<blockquote>
<p>“<em>secure_file_priv can be set to NULL to disable all import and export operations.</em></p>
<p><em>The server checks the value of secure_file_priv at startup and writes a warning to the error log if the value is insecure. A non-NULL value is considered insecure if it is empty, or the value is the data directory or a subdirectory of it, or a directory that is accessible by all users. If secure_file_priv is set to a nonexistent path, the server writes an error message to the error log and exits.</em></p>
<p><em>Previously, the secure_file_priv system variable was empty by default. Now the default value is platform specific and depends on the value of the INSTALL_LAYOUT CMake option, as shown in the following table.</em>”</p>
</blockquote>
<p>Many of the forum posts I found were written before this change and were horribly confusing. The default value of secure_file_priv for the Homebrew install of MySQL 5.7.17 was NULL, which prevented me from writing with outfile entirely. To change secure_file_priv I first had to find the configuration file. According to my research it was named my.cnf and located in one of dozens of different places depending on who you believe. The forum posts I found were split between many versions of MySQL and various operating systems.</p>
<h3 id="the-solution">The Solution</h3>
<p>I found the <a href="http://stackoverflow.com/questions/7973927/for-homebrew-mysql-installs-wheres-my-cnf">secret</a>! Just ask mysql where it was looking for my.cnf.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mysql <span class="nt">--help</span> | more
</code></pre></div></div>
<p>Which returned in part the following list of locations:</p>
<blockquote>
<p>“<em>Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf /usr/local/etc/my.cnf ~/.my.cnf</em>”</p>
</blockquote>
<p>The Homebrew install does not create a my.cnf by default, so I had to make one of my own. I decided to put the configuration file in my user directory and found this <a href="https://github.com/piwik/piwik/issues/9528">helpful tip</a> on what to put in it.</p>
<p>After creating ~/.my.cnf with the following contents, I was ready to try creating my outfile again.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[mysqld]
secure_file_priv = ''
</code></pre></div></div>
<p>At terminal prompt:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mysql.server restart
</code></pre></div></div>
<p>In mysql:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mysql</span><span class="o">></span> <span class="k">select</span> <span class="o">*</span> <span class="k">from</span> <span class="n">reportSources</span> <span class="k">into</span> <span class="n">outfile</span> <span class="s1">'/Users/paulmccombs/reportSources.txt'</span><span class="p">;</span>
<span class="n">Query</span> <span class="n">OK</span><span class="p">,</span> <span class="mi">12</span> <span class="k">rows</span> <span class="n">affected</span> <span class="p">(</span><span class="mi">0</span><span class="p">.</span><span class="mi">04</span> <span class="n">sec</span><span class="p">)</span>
</code></pre></div></div>
<p>Success!!!</p>
<p><a name="myfootnote1"><sup>1</sup></a> Note here I am starting MySQL manually because I don’t intend to have the server running on my computer all of the time. Just for the one project. So there is no need for me to set it up to auto start.</p>
Exploring Relations in QGIS2015-11-11T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2015/11/11/QGIS-Relations<p>Last night I shared a <a href="http://www.lazym8.com/datawrangler/qgis_relations">presentation on Relations in QGIS 2.10</a> at the November <a href="http://www.meetup.com/Puget-Sound-QGIS-Users-Group/">Meetup</a> of the <a href="http://psqgis.org">Puget Sound QGIS User’s Group</a>.</p>
<p>The Puget Sound QGIS User’s Group is a great group where we get together and learn how to use the various features of QGIS. We all pitch in to share what we know and learn more in a fun atmosphere.</p>
Make A Custom Symbol for QGIS2015-08-09T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2015/08/09/Custom-SVG_Marker<p>This is a document describing what I had to learn to create a custom point
symbol to use in <a href="http://qgis.org/en/site/">QGIS</a>. This process involved some
workarounds that appear to be unique to Apple OSX 10.10.</p>
<p>In order to accomplish this task I needed to:</p>
<ul>
<li>Do some research<sup id="fnref:research" role="doc-noteref"><a href="#fn:research" class="footnote" rel="footnote">1</a></sup>.</li>
<li>Have QGIS installed on your computer.
<ul>
<li>I am running version 2.8.2<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">2</a></sup></li>
</ul>
</li>
<li>Have software capable of creating an SVG file.
<ul>
<li>In my case I chose <a href="https://inkscape.org/en/">InkScape</a> is an open
source vector drawing program that natively creates SVG files.</li>
<li>In order to use Inkscape with OSX 10.10<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">3</a></sup>, first install
<a href="http://xquartz.macosforge.org/landing/">XQuartz</a>. I had to overcome a
bug where InkScape will not launch<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">4</a></sup>.</li>
</ul>
</li>
<li>Import or draw the design that you want to use for your custom symbol.
<ul>
<li>In my case I’m trying to create the
<a href="http://www.lazym8.com/lazym8.gif">brand</a> of my grandfather’s ranch.
Which so far looks very rough, but the possibilities are promising.</li>
</ul>
</li>
<li>Save the drawing out as a
<a href="https://en.wikipedia.org/wiki/Scalable_Vector_Graphics">SVG</a> file.</li>
<li>Alter the SVG file so QGIS can control the fill color the border color and
the border width.
<ul>
<li>Do more research<sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">5</a></sup></li>
<li>Open the SVG file in a text editor.</li>
<li>Find the line in your saved SVG file that reads
<code class="language-plaintext highlighter-rouge">style="fill:#4d4d4d;stroke:#000000;stroke-width:1;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"</code></li>
<li>Delete that line.</li>
<li>insert the following line in it’s place.</li>
<li><code class="language-plaintext highlighter-rouge">fill="param(fill) #FFF" stroke="param(outline) #000" stroke-width="param(outline-width) 1"</code></li>
</ul>
</li>
<li>Use your new marker symbol in QGIS.
<ul>
<li>Open the <em>Layer Properties</em> window for a point vector layer.</li>
<li>Change to the <em>Style</em> tab.</li>
<li>Expand the single listed <em>Marker</em> symbol to see the <em>Simple marker</em>
layer.</li>
<li>Click to highlight the <em>Simple marker</em>.</li>
<li>Change the <em>Symbol layer type</em> to <em>SVG marker</em>.</li>
<li>Look for a text box containing a full path and file name. Click the
button to the right with three dots.</li>
<li>Browse to the SVG file just edited and <em>Open</em> it.</li>
<li>Now the various attributes are available to adjust.</li>
</ul>
</li>
<li>Add custom SVGs to the QGIS interface.
<ul>
<li>Collect all the SVGs you want to make available into a directory.</li>
<li>Open the <em>Options</em> window from the <em>Settings</em> menu.</li>
<li>Select the <em>System</em> tab.</li>
<li>Click the <em>Add</em> button above the
<em>Path(s) to search for Scalable Vector Graphic (SVG) symbols</em>
list box.</li>
<li>Browse to the directory where you have placed your custom SVGs.</li>
<li>Click the <em>Choose</em> button.</li>
<li>Now you will see your custom symbols as choices when you go into the
<em>SVG marker</em> window.</li>
</ul>
</li>
</ul>
<p><strong>Footnotes</strong></p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:research" role="doc-endnote">
<p><a href="http://gis.stackexchange.com/questions/32982/how-to-add-my-own-symbols-to-the-single-marker-or-svg-marker-selection-list">StackExchange</a>
is often the first step in me learning how to do something. <a href="#fnref:research" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:1" role="doc-endnote">
<p>The most current OSX installer is available from
<a href="http://www.kyngchaos.com/software/qgis">Kyng Chaos</a>. Be sure to install
the packages listed under Requirements. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Download <a href="https://inkscape.org/en/download/mac-os/">Inkscape for OSX</a>. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>See the
<a href="http://www.inkscapeforum.com/viewtopic.php?f=22&t=19119">InkScape forum</a>
for a discussion of the problem. I was not able to get the solutions
discussed to work. I found that I could launch the executable that is
hidden inside the .app bundle. If Inkscape is installed in the
<em>Applications</em> directory, the application can be launched at the command
line by typing “<code class="language-plaintext highlighter-rouge">/Applications/Inkscape.app/Contents/MacOS/Inkscape</code>”. To
simplify things for myself I added that command as an item on the X11
<em>Applications</em> menu. So I can start InkScape by launching the X11 app
installed from XQuartz, opening the <em>Applications</em> menu, and choosing the
InkScape item I added. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>Found the basic <a href="http://blog.sourcepole.ch/2011/06/30/svg-symbols-in-qgis-with-modifiable-colors/">instructions</a>
and some more <a href="http://gis.stackexchange.com/questions/45180/how-to-create-svg-symbols-that-have-modifiable-fill-color-stroke-color-and-stro">details</a>
for Inkscape generated SVG files.
<a href="http://gis.stackexchange.com/users/7348/jgrocha">jgrocha</a> provided the
clearest answer and the one that put it all together. <a href="#fnref:4" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Let's Access Some Proprietary Data Formats2015-07-17T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2015/07/17/proprietary<p>Working to solve a image processing problem, I found this enticing <a href="http://briantjacobs.com/mrsid-gdal-homebrew/">tutorial</a> from Brian T. Jacobs.</p>
<p>It’s instructions to compile <a href="http://www.gdal.org/">GDAL</a> for OSX using <a href="http://brew.sh/">Homebrew</a> so it can read <a href="https://en.wikipedia.org/wiki/MrSID">MrSID</a> compressed images and <a href="http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//003n00000007000000">ESRI’s File Geodatabase</a> vector data format.</p>
<p>I’ve had really nice experiences installing software with Homebrew, but I understand that if you already using MacPorts there can be conflicts.</p>
First Post2015-07-17T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2015/07/17/First-Post<p>Experimenting with creating a blog using Github pages. I’m not sure how it will work out.</p>
<p>I intend to write about useful tips and discoveries I make for working with geodata and open source software.</p>
<p>Thanks are due to Jonathan McGlone whose great <a href="http://jmcglone.com/guides/github-pages/">tutorial</a> I used to get this blog up and running.</p>
<p>Updated 11 Nov 2015:
I’m posting some older items migrated from the <a href="www.lazym8.com">Lazy M 8 page</a>. Just incase you were wondering why this isn’t listed as the first post.</p>
Puget Sound QGIS User's Group2015-07-07T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2015/07/07/PSQGIS<p>Some beginning users of <a href="http://www.qgis.org/en/site/index.html">QGIS</a> including myself, have started up a <a href="http://www.meetup.com/Puget-Sound-QGIS-Users-Group/">Puget Sound QGIS User’s Group</a>.</p>
<p>We’re just starting out. If you’re interested in taming the power of GIS with open source software, QGIS is for you. It runs on Java, supporting Windows, Mac, and Linux.</p>
2014 FOSS4G Notes2014-09-30T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2014/09/30/fos4g<p>I attended the FOSS4G<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> <a href="http://2014.foss4g.org/">2014 conference</a> in Portland Oregon.</p>
<p>It was a lot of fun and packed with great information. I’ve published my <a href="http://mccombsp-kingco.github.io/">notes on GitHub</a>.</p>
<p><strong>Footnotes</strong></p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>Free and Open Source Software for GeoSpatial conference is held annually by <a href="http://www.osgeo.org/">OSGeo</a>. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Correct Addressing Is In The Details2014-08-10T00:00:00+00:00http://geodatawrangler.lazym8.com/blog/2014/08/10/addressing<p>Check out the <a href="http://www.meetup.com/OpenStreetMap-Seattle">OpenStreetMap Seattle</a> meetups, they’re a ton of fun.</p>
<p>Yesterday, I gave a short presentation at their OpenStreetMap 10th Anniversary Event. Check it Out: <a href="http://www.lazym8.com/datawrangler/AddressingDetails">Correct Addressing Is In The Details</a></p>