tag:blogger.com,1999:blog-6798545106897711842024-03-08T02:07:22.200-08:00David Grayson's blogThis is my professional blog, where I share interesting new things I have learned about computer programming.DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.comBlogger27125tag:blogger.com,1999:blog-679854510689771184.post-34362801918609670592017-03-25T12:04:00.001-07:002017-04-12T07:12:00.743-07:00One-part AVR-controlled "boost converter"<p>I built a crappy AVR-controlled boost converter with just one part (an inductor) that can blink a blue LED when you power it at 2.3 V. See the video here:</p>
<div style="position:relative;height:0;padding-bottom:56.25%"><iframe src="https://www.youtube.com/embed/Igi9MfjAJ7Q?ecver=2" width="640" height="360" frameborder="0" style="position:absolute;width:100%;height:100%;left:0" allowfullscreen></iframe></div>
<p>I did this because there was a <a href="https://wp.josh.com/2017/03/22/challenge-design-an-avr-controlled-boost-converter-using-only-one-part/comment-page-1/#comment-8937">challenge</a> on a cool electronics blog that I follow.</p>
<p>The "boost converter" is pretty crappy because it does not produce a stedy voltage and it is only useful when the input voltage is between 2.3 V and 2.5 V. Below that, my AVR ATmega328P seems to stop running. Above that, the blue LED is on all the time (though it does get noticeably brighter when I run the boost).</p>
<p>In this system, VCC is only boosted by about 0.7 V; it can go a little bit higher if I use more I/O lines, but it can't go much higher than that because of the blue LED. If I take the blue LED away, I see shorter but more extreme boosts on VCC. With one I/O line, I see a boost of 1.2 V. With two I/O lines, I see a boost of 1.7 V, and it seems like the sudden boost causes issues for the AVR and it usually resets after one or two boosts. With three I/O lines, I see a very sharp boost of 2.0 V for 2 us, and it consistently causes the AVR to reset.</p>
<p>
<b>Update: I received a shirt from Josh for winning the contest. I like it!</b>
</p>
<a href="http://www.davidegrayson.com/for_blog/entry/20170325_boost/josh_boost_converter_winner.jpg"><img alt="The shirt is gray, has a picture of an inductor, and says: Winner! Josh.com Boost Converter Design Contest" src="http://www.davidegrayson.com/for_blog/entry/20170325_boost/josh_boost_converter_winner_thumb.jpg"/></a>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1tag:blogger.com,1999:blog-679854510689771184.post-65629681855370832872016-10-21T11:08:00.000-07:002016-10-21T13:58:07.110-07:00GitHub.com hosts file<p>GitHub and other popular sites are <a href="https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/">suffering from a DDoS attack</a> right now. In case you actually want to use GitHub, here are entries you can add to your system's <a href="https://en.wikipedia.org/wiki/Hosts_(file)">hosts file</a>:</p>
<pre>
192.30.253.113 github.com
103.245.222.133 assets-cdn.github.com
</pre>
<p>Just remember to <b>remove these entries later</b> once GitHub starts working again, since those IP addresses might change in the future without notice.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1tag:blogger.com,1999:blog-679854510689771184.post-81660233493474442932016-06-22T19:04:00.000-07:002016-06-22T19:05:25.695-07:00Detailed teardown of a Roomba Model 550 robot<a href="https://imgur.com/a/qprME"><img src="http://i.imgur.com/iCXr3Vi.jpg" alt="" style="max-width: 90%;"/></a>
<p>I recently bought a Roomba for $5 from my neighbor at a garage sale. I tore it down in order to see what is going on inside it and learned a lot from it. Check out <b><a href="https://imgur.com/a/qprME">my detailed photo album on imgur</a></b>.</p>
DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-44462360132734871042016-02-24T08:48:00.000-08:002016-02-24T10:18:29.179-08:00Windows 32-bit structured exception handling example for GCC<p>I recently spent a few hours learning about structured exception handling (SEH) in 32-bit (i686) Microsoft Windows. The canonical article documenting this seems to be <a href="https://www.microsoft.com/msj/0197/exception/exception.aspx">A Crash Course on the Depths of Win32™ Structured Exception Handling</a> by Matt Pietrek in 1997.</p>
<p>After reading the first 1600 words of the article, I came across a code sample (Figure 3) that shows the most basic way one can use structured exception handling. The code uses C++ and a little bit of inline assembly to add its own exception handler to the beginning of the linked list of exception handlers that is stored in the <a href="https://en.wikipedia.org/wiki/Win32_Thread_Information_Block">Thread Environment Block</a>. It then generates an segmentation fault (by writing to address 0) in order to demonstrate that the exception handler can run, fix the problem, and tell the OS to resume and try the problematic instruction again.</p>
<p>This example did not compile in my favorite C++ development environment for Windows, which is <a href="http://msys2.github.io/">MSYS2</a>. MSYS2 comes with a <a href="http://mingw-w64.org/doku.php">mingw-w64</a> GCC compiler. I don't know much about x86 assembly but I managed to port the example and get it to run. The main thing I had to change was the syntax of the inline assembly. Here is the code:</p>
<pre><code class="language-c">
/* Basic demonstration of Win32 structured exception handling (SEH).
This code is based on Figure 3 (MYSEH.CPP) from this article:
https://www.microsoft.com/msj/0197/exception/exception.aspx
Originally written by Matt Pietrek for the January 1997 Microsoft
Systems Journal.
Modified in 2016 by David E. Grayson to work with gcc.
*/
#include <windows.h>
#include <stdio.h>
DWORD scratch = 10;
EXCEPTION_DISPOSITION __cdecl
_except_handler(
struct _EXCEPTION_RECORD * ExceptionRecord,
void * EstablisherFrame,
struct _CONTEXT * ContextRecord,
void * DispatcherContext)
{
// Indicate that we made it to our exception handler.
printf("Hello from an exception handler\n"); fflush(stdout);
// Change EAX in the context record so that it points to some
// place where we can successfully write.
ContextRecord->Eax = (DWORD_PTR)&scratch;
// Tell the OS to restart the faulting instruction.
return ExceptionContinueExecution;
}
int main()
{
__asm__( // Build EXCEPTION_REGISTRATION record:
"push %0\n" // Bytes 4 to 7: Address of handler function
"push %%fs:0\n" // Bytes 0 to 3: Address of previous handler
"movl %%esp, %%fs:0\n" // Install new EXECPTION_REGISTRATION
:
: "r" ((DWORD_PTR)_except_handler)
);
__asm__(
"movl $0, %eax\n" // Zero out EAX
"movl $1, (%eax)\n" // Write to EAX to deliberately cause a fault
);
printf("After writing!\n"); fflush(stdout);
__asm__( // Remove our EXCEPTION_REGISTRATION record
"movl (%esp), %eax\n" // Get pointer to previous record
"movl %eax, %fs:0\n" // Install previous record
"addl $8, %esp\n" // Clean our EXCEPTION_REGISTRATION off stack
);
printf("scratch = %d\n", scratch);
return 0;
}
</code></pre>
<p>I compiled the code in a MinGW-w64 32-bit Shell, with <code>/mingw32/bin/g++</code>, using the command <code>g++ -g3 -O0 myseh.cpp</code>. When I run <code>a.exe</code>, the output is:</p>
<pre>
Hello from an exception handler
After writing!
scratch = 1
</pre>
<p>So this shows that our exception handler really did work!</p>
<h2>Using less assembly</h2>
<p>The code above worked, but I would not use it in production because it contains inline assembly that does sketchy things to the stack and registers without telling the compiler about it. We can make it a little bitter by using functions and structs provided by Windows headers to manipulate the thread environment block (which were probably not available in 1997 when Matt Pietrek wrote his article).</p>
<pre><code class="language-c">
/* Basic demonstration of Win32 structured exception handling (SEH).
This code is based on Figure 3 (MYSEH.CPP) from this article:
https://www.microsoft.com/msj/0197/exception/exception.aspx
Originally written by Matt Pietrek for the January 1997 Microsoft
Systems Journal.
Modified in 2016 by David E. Grayson to work with gcc and use less assembly.
*/
#include <windows.h>
#include <winternl.h>
#include <stdio.h>
DWORD scratch = 10;
EXCEPTION_DISPOSITION __cdecl
_except_handler(
struct _EXCEPTION_RECORD * ExceptionRecord,
void * EstablisherFrame,
struct _CONTEXT * ContextRecord,
void * DispatcherContext)
{
// Indicate that we made it to our exception handler.
printf("Hello from an exception handler\n"); fflush(stdout);
// Change EAX in the context record so that it points to some
// place where we can successfully write.
ContextRecord->Eax = (DWORD_PTR)&scratch;
// Tell the OS to restart the faulting instruction.
return ExceptionContinueExecution;
}
int main()
{
NT_TIB * teb = (NT_TIB *)NtCurrentTeb();
// Install our handler at the beginning of the list.
EXCEPTION_REGISTRATION_RECORD reg;
reg.prev = teb->ExceptionList;
reg.handler = (PEXCEPTION_ROUTINE)_except_handler;
teb->ExceptionList = &reg;
__asm__(
"movl $0, %%eax\n" // Zero out EAX
"movl $1, (%%eax)\n" // Write to EAX to deliberately cause a fault
::: "memory", "eax"
);
printf("After writing!\n"); fflush(stdout);
// Remove our handler from the list.
teb->ExceptionList = reg.prev;
printf("scratch = %d\n", scratch);
return 0;
}
</code></pre>
<p>We have to use two casts in the code above. First, we cast the output of <code>NtCurrentTeb()</code> because the struct type it points to by default is not very useful and just has a lot of fields with names like <code>Reserved1</code>. Second, we cast <code>_except_handler</code> because there is a disagreement about whether the exception handler should use the <code>__cdecl</code> calling convention or <code>__stdcall</code>. Both calling conventions seem to work, but since the calling convention of our <code>_except_handler</code> function does not match the calling convention of the <code>handler</code> member of the struct, we need to use a type-cast to satisfy the type-checking the GCC does at compile time. I have noticed <a href="https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-devcon-git/03-stdcall-callback.patch">cases</a> like this in the past, where I had to change the calling convention of something to get it to compile with GCC, but presumably it would compile fine with the Microsoft C compiler. The function prototype of <code>_except_handler</code> in Microsoft's headers uses <code>__cdecl</code>, while the type definition of <code>EXCEPTION_ROUTINE</code> uses <code>NTAPI</code>, which means <code>__stdcall</code>.</p>
<h2>What about 64-bit?</h2>
<p>Apparently structured exception handling for 64-bit Windows is done in a totally different way, so you can't easily adapt the code above to work in 64-bit mode. More info about <a href="https://www.osronline.com/article.cfm?article=469">x64 Structured Exception Handling</a> is available from OSR Online.</p>
<h2>What's the point?</h2>
<p>Microsoft's C compiler emits code similar to what I have above when you use the <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/ms680144(v=vs.85).aspx">__try, __except, and __finally</a> keywords. GCC does not yet support these keywords, but Clang has <a href="http://clang.llvm.org/docs/MSVCCompatibility.html">partial support</a> for them. I have heard (but not verified) that <a href="https://webkit.org/">WebKit</a> uses SEH, so if we want to compile WebKit with GCC, someone needs to add SEH to GCC first. Or maybe we could compile WebKit but we would have to somewhat disable its sandboxing features. I haven't actually tried to compile WebKit with GCC yet.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1tag:blogger.com,1999:blog-679854510689771184.post-12148159460350609912015-10-09T10:45:00.003-07:002015-10-09T10:45:29.982-07:00From Pololu: Sumo ring border angle detection<p>I figured out something cool you can do with a robot and wrote about it on the Pololu blog: <a href="https://www.pololu.com/blog/543/sumo-ring-border-angle-detection"><b>Sumo ring border angle detection</b></a>.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-5926596619014313982015-06-02T20:32:00.000-07:002015-06-02T20:32:13.381-07:00USB CDC ACM braindump<p>I just dumped a lot of stuff I know about USB CDC ACM devices into a <b><a href="http://stackoverflow.com/a/30610224/28128">StackOverflow answer</a></b>. Check it out if you want to learn more about the standard way to make a USB device that has a virtual serial port.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-29020482146139074942015-05-29T18:48:00.000-07:002015-05-29T18:48:06.453-07:00Ways to return a string from a C function<p>I am working on a C library and I need to have a flexible, easy, efficient way to return strings to the user. See my gist on this topic: <a href="https://gist.github.com/DavidEGrayson/db13d38cbe5c95db64c4">https://gist.github.com/DavidEGrayson/db13d38cbe5c95db64c4</a>.</p>
DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-35491591779093406332015-03-02T20:59:00.003-08:002015-03-02T21:08:06.912-08:00Non-bitcoin assets on the block chain are not interesting<p>I frequently hear people in the <a href="http://reddit.com/r/bitcoin">Bitcoin community</a> say that money is just the first application of the Bitcoin block chain, and that they want to use it for tracking other assets like land and securities. That thinking is misguided and I would like to explain why.</p>
<p>First of all, let's talk about the data in the block chain. The payload data for the block chain is mainly just a bunch of blobs of structured binary data called transactions. Each transaction has a list of <i>inputs</i> and a list of <i>outputs</i>, and it specifies how much value is stored inside each output. The value inside each output is just a positive integer which represents how many bitcoins are in that output. Each transaction output also has a script which determines who can unlock the output, and the typical script will contain something called a Bitcoin address. The rest of the data in the block chain, and all the software that runs on the nodes in the Bitcoin network, is designed to fulfill one purpose: it establishes a decentralized consensus on the set of unspent transaction outputs. It does this in a way that anyone can use the network, but nearly no one can modify its data maliciously.</p>
<p>The block chain, and the servers that maintain it, know nothing about real estate or stocks. So when someone tells you they have developed a way to represent non-bitcoin assets on the block chain, what they are saying is that they have invented a set of rules for how to interpret the data in the block chain in order to add extra information to it about asset ownership.</p>
<p>These rules that they invented are not enforced in the block chain. Therefore, if you are an owner of a non-bitcoin block chain asset, then you have to trust some third party to actually give value to your asset. That third party could arbitrarily change the rules of the game some day and declare that your assets are not valid, in a very similar manner to how PayPal or Coinbase can arbitrarily freeze your account. The block chain doesn't care, so the only recourse you would have is to complain loudly in public and/or file a lawsuit. There is nothing you can actually do by yourself in order to maintain control of that asset. Therefore, that kind of asset is inferior to an actual, real bitcoin, and it is not a particularly interesting thing to talk about.</p>
<p>The only benefit I can see to these kind of schemes is that there would be more transparency in the ownership and transfer of these assets. But you don't need the block chain to achieve that. Imagine this alternative system instead: the central authority that everyone is trusting could simply maintain a centralized ledger themselves. They could publish a new version of the ledger on their website every hour, signed with their company's PGP key. The asset holders could use ECDSA cryptography, or any kind of authentication mechanism that the company implements, in order to authenticate themselves and issue orders for transferring the asset. This kind of system is totally possible with today's technology, and it would be cheaper to implement than a block chain-based system because you wouldn't have to download tens of gigabytes of data or pay fees to Bitcoin miners.</p>
<p>So why does everyone keep talking about non-bitcoin assets on the block chain and writing <a href="https://en.bitcoin.it/wiki/Coinprism">software</a> and <a href="https://en.bitcoin.it/wiki/Colored_Coins">protocols</a> for it? There are people who have spent a lot more time thinking about this than I have, so maybe I am missing something. Please enlighten me in the comments.</p>
<p><i>TLDR: The bitcoin network establishes a decentralized consensus on the set of unspent transaction outputs. It doesn't know anything about non-bitcoin assets, so you'd have to trust a third party in order to actually hold such an asset. If you are trusting a third party, there is no real reason to use the block chain.</i></p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com9tag:blogger.com,1999:blog-679854510689771184.post-460565711319833482015-02-15T12:12:00.003-08:002015-03-04T19:48:58.198-08:00Ruby blocks with brackets are parsed differently<p>In <a href="https://www.ruby-lang.org/">Ruby</a>, you can write blocks using either brackets or with the do/end keywords. Check out this code that gives surprising results:</p>
<pre><code class="language-ruby">
a = []
a.concat a.map { |b| } # => []
a.concat a.map do |b|
end
# => no implicit conversion of Enumerator into Array (TypeError)
</code></pre>
<p>It turns out that the Ruby parser actually cares which notation you use for your blocks, and that affects which method will receive the block. In the code above, a block written with brackets gets passed to the last method in the statement. A block written with do/end gets passed to the first method, which causes the code example above to break.</p>
<p>You can even use both types of blocks in the same statement, and they go to different methods:</p>
<pre><code class="language-ruby">
def f(x)
puts 'f got block' if block_given?
end
def g(x)
puts 'g got block' if block_given?
end
def h
puts 'h got block' if block_given?
end
f g h { } do
end
# f got block
# h got block
</code></pre>
<p>I used to think that the different notations for blocks were just equivalent ways of writing the same thing, and I said that do/end should be used if and only if the block is multi-line. Now it seems like there might be legitimate reasons to break that rule, or to come up with a different rule.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com2tag:blogger.com,1999:blog-679854510689771184.post-55645909447940597222014-09-22T12:26:00.002-07:002014-09-22T12:27:03.838-07:00Introducing RPicSim<p>Check out the post I recently wrote on the Pololu blog: <a href="http://www.pololu.com/blog/286/introducing-rpicsim"><b>Introducing RPicSim</b></a>. It talks about a tool we made for using Ruby, RSpec, and test-driven development to aid the development of PIC microcontroller firmware.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-64472296666468948882014-07-11T10:11:00.000-07:002014-07-11T10:12:42.061-07:00Version 2.0.0 of minimu9-ahrs releasedThis week I released version 2.0.0 of minimu9-ahrs, which adds support for the MinIMU-9 v3. You can see more information <a href="http://www.pololu.com/blog/376/version-2.0.0-of-minimu9-ahrs-released">on the Pololu blog</a>.DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1tag:blogger.com,1999:blog-679854510689771184.post-63326160430247068602014-04-06T16:32:00.000-07:002015-03-04T19:47:14.224-08:00Introducing the Ruby ECDSA gem<p>I am happy to announce that I released version 1.0.0 of the <a href="https://github.com/DavidEGrayson/ruby_ecdsa"><b>Ruby ecdsa gem</b></a> today. ECDSA stands for Elliptic Curve Digital Signature Algorithm. This algorithm, defined in a document called <a href="http://www.secg.org/collateral/sec1_final.pdf">SEC1</a>, allows you to make secure, cryptographic digital signatures of messages. The ecdsa gem is an implementation of ECDSA written almost entirely in pure Ruby.</p>
<p>To install the gem, run:</p>
<pre>
gem install ecdsa
</pre>
<p>One of the main purposes of the ecdsa gem is education; it is a good resource for people who want to learn about ECDSA. The ecdsa gem seems to be a lot slower than OpenSSL, but that is okay. Verifying a single secp256k1 signature with the gem takes about 280 ms on my machine.</p>
<p>The main reason that I made the ecdsa gem is because I wanted to do some experiments related to <a href="https://bitcoin.org/">Bitcoin</a>. My language of choice for experimentation is Ruby. Ruby has built-in support for ECDSA through its <a href="http://ruby-doc.org/stdlib-2.0/libdoc/openssl/rdoc/OpenSSL.html">OpenSSL binding</a>, but I found it frustrating to use.</p>
<h2>Comparison to OpenSSL</h2>
<p>Ruby's built-in OpenSSL ECDSA support is frustrating mainly because the documentation is very sparse. For example, it says on <a href="http://ruby-doc.org/stdlib-2.0/libdoc/openssl/rdoc/OpenSSL/PKey/EC/Point.html">this page</a> that you can create a new OpenSSL::PKey::EC::Point object from a "group" and a "bn". The group part is easy to figure out, but the "bn" part is not obvious at all. The "bn" stands for "big number". In general, a point can be created from a private key, which is a big number, so I would think that the "bn" argument might be the private key. Instead, the "bn" turns out to actually be a binary representation of the two coordinates of the public key, as defined in section 2.3.4 of <a href="http://www.secg.org/collateral/sec1_final.pdf">SEC1</a>. There was no way to figure this out from the Ruby documentation or the OpenSSL documentation; I had to do an experiment to verify it after someone on StackOverflow <a href="http://stackoverflow.com/questions/22293864/ruby-openssl-convert-elliptic-curve-point-octet-string-into-opensslpkeyec">explained</a> it to me.</p>
<p>In contrast, the Ruby ecdsa gem documentation explains the types of all its parameters and cites official documents when appropriate. Also, the classes and routines for doing cryptography are very clearly separated from the routines for encoding and decoding data from strings. All the binary formatting routines live in the <code>ECDSA::Format</code> module, and the parts of the gem that do actual cryptographic calculations do not depend on them.
</p>
<p>Here is a bit of Ruby code showing how you would verify an ECDSA signature using OpenSSL and then again using the ECDSA gem:</p>
<pre><code class="language-ruby">
digest =
"\xbf\x91\xfb\x0b\x4f\x63\x33\x77\x4a\x02\x2b\xd3\x07\x8e\xd6\xcc" \
"\xd1\x76\xee\x31\xed\x4f\xb3\xf9\xaf\xce\xb7\x2a\x37\xe7\x87\x86"
signature_der_string =
"\x30\x45" \
"\x02\x21\x00" \
"\x83\x89\xdf\x45\xf0\x70\x3f\x39\xec\x8c\x1c\xc4\x2c\x13\x81\x0f" \
"\xfc\xae\x14\x99\x5b\xb6\x48\x34\x02\x19\xe3\x53\xb6\x3b\x53\xeb" \
"\x02\x20" \
"\x09\xec\x65\xe1\xc1\xaa\xee\xc1\xfd\x33\x4c\x6b\x68\x4b\xde\x2b" \
"\x3f\x57\x30\x60\xd5\xb7\x0c\x3a\x46\x72\x33\x26\xe4\xe8\xa4\xf1"
public_key_octet_string =
"\x04" \
"\xfc\x97\x02\x84\x78\x40\xaa\xf1\x95\xde\x84\x42\xeb\xec\xed\xf5" \
"\xb0\x95\xcd\xbb\x9b\xc7\x16\xbd\xa9\x11\x09\x71\xb2\x8a\x49\xe0" \
"\xea\xd8\x56\x4f\xf0\xdb\x22\x20\x9e\x03\x74\x78\x2c\x09\x3b\xb8" \
"\x99\x69\x2d\x52\x4e\x9d\x6a\x69\x56\xe7\xc5\xec\xbc\xd6\x82\x84"
# Verifying with openssl.
require 'openssl'
ec = OpenSSL::PKey::EC.new('secp256k1')
key_bn = OpenSSL::BN.new(public_key_octet_string, 2) # 2 means binary
ec.public_key = OpenSSL::PKey::EC::Point.new(ec.group, key_bn)
result = ec.dsa_verify_asn1(digest, signature_der_string)
puts result # => true
# Verifying with the new ecdsa gem, version 1.0.0.
require 'ecdsa'
group = ECDSA::Group::Secp256k1
point = ECDSA::Format::PointOctetString.decode(public_key_octet_string, group)
signature = ECDSA::Format::SignatureDerString.decode(signature_der_string)
result = ECDSA.valid_signature?(point, digest, signature)
puts result # => true
</code></pre>
<p>As you can see above, OpenSSL requires you to make this weird <code>OpenSSL::PKey::EC</code> object which represents an elliptic curve but it also holds a reference to a public key, which is a point on the curve. The object is weird because you cannot say concisely what it really is using the vocabulary of the problem domain, which is ECDSA. In contrast, the ecdsa gem uses the <code>ECDSA::Group</code> class to represent elliptic curves and the <code>ECDSA::Point</code> class to represent points on those curves.</p>
<p>You will also see that OpenSSL's verification method takes an ASN1 string directly, instead of taking a more abstract object that represents a signature. As a result, it is not easy for the user to inspect the internal structure of that signature, which actually consists of two big numbers. In contrast, the ecdsa gem takes a <code>ECDSA::Signature</code> object, so you are not forced to use any particular binary format to represent the signature and it is easier to see what is going on.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com12tag:blogger.com,1999:blog-679854510689771184.post-46157601714126786252014-01-30T10:14:00.001-08:002015-03-04T19:46:03.248-08:00What Ruby does at parse time<p>Yesterday at the <a href="http://www.meetup.com/las-vegas-ruby-on-rails/events/159460712/">Las Vegas Ruby Group</a>, someone was surprised that <a href="http://ruby-doc.org/core-2.1.0/Kernel.html#method-i-global_variables">Kernel#global_variables</a> will list global variables that have not been used yet. The same thing is actually true for symbols and methods, as shown here:</p>
<pre><code class="language-ruby">
p Symbol.all_symbols.map(&:to_s).grep(/neve/)
# => ["never", "never_runs", "$never_global", "never_sym"]
p global_variables.grep(/neve/) # => ["$never_global"]
def never_runs
$never_global
:never_sym
end
</code></pre>
<p>The first line of this program tells us that Ruby has already found four things. It found a symbol <code>:never</code> which apparently comes from the Ruby interpreter since we never defined it. It has found the name of the method in our program even though we haven't defined that method yet. It found a symbol we used in that method even though we are never going to call the method. Similarly, it found a the name of a global variable we were never going to use.</p>
<p>The second line just reproduces the original thing that happened at the Ruby group, and shows that Ruby is aware of "$never_global" as an actual global variable and not just a symbol.</p>
<p>To understand this, you need to know that Ruby does not simply run a source file one line at a time:</p>
<ul>
<li>First, Ruby parses the syntax of all the code in the file.</p>
<li>Later, Ruby runs the code in the file.</p>
</ul>
<p>We already intuitively know this because whenever we make a syntax error in a file, we can see that Ruby just reports a syntax error and does not run any of the other code in the file. This means it is parsing the entire file before running any of it.</p>
<p>So it shouldn't be surprising that the Ruby interpreter is aware of symbols and global variables before they are used; it just tells us that those things are probably recorded <i>at parse time instead of run time</i>. Some more evidence for this conclusion can be found in the fact that the Ruby interpreter has lots of <a href="https://github.com/ruby/ruby/blob/v2_1_0/parse.y#L10137-10763">symbol table code in parse.y</a>.</p>
<p>Most of the time you can just pretend that Ruby processes your file one line at a time, but this is a case where the details of its behavior reveal a little bit about how it actually works.</p>
<p><i>The code in this blog post was executed with Ruby 2.0.0. Ruby 1.8.7 had the same results except the symbol "never" was not present.</i></p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-52709171339977427352013-11-30T18:45:00.000-08:002013-12-02T09:01:04.070-08:00Running JRuby under JDB in Windows<p>I am working on some <a href="http://jruby.org/">JRuby</a> code that calls third-party Java libraries. The Java libraries were behaving oddly so I wanted to run them under <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/jpda/jdb.html">JDB</a>, the Java Debugger, so I can step through them line by line and inspect important values.</p>
<p>In Linux and Mac OS X, I think you should be able to do this by passing the <code>--jdb</code> option to JRuby when you run it. In Windows, JRuby does not support that option, but I was able to figure out how to do it anyway. Here is the command I used:</p>
<pre>
"C:/Program Files/Java/jdk1.7.0_45/bin/jdb.exe" -classpath "C:/jruby-1.7.4/lib/jruby.jar" -Djruby.home="C:/jruby-1.7.4" org.jruby.Main
</pre>
<p>The <code>-classpath</code> argument to JDB allows it to find the jruby.jar file, which defines jruby's org.jruby.Main class, which is the class to start if you want to use JRuby. The <code>-Djruby.home</code> option is recognized by JRuby and allows it to find its libraries and gems. If you want to pass any arguments to JRuby, then you can add them on to the end of this command, after org.jruby.Main. For my situation, I wanted to run <a href="http://rspec.info/">RSpec</a> so I added <code>-S rspec</code>:</p>
<pre>
"C:/Program Files/Java/jdk1.7.0_45/bin/jdb.exe" -classpath "C:/jruby-1.7.4/lib/jruby.jar" -Djruby.home="C:/jruby-1.7.4" org.jruby.Main -S rspec
</pre>
<p>Once you have started JDB successfully, you should see a message like "Initializing jdb ..." and a command prompt on the next line. My basic workflow is that I would figure out what class and method I want to inspect, and then I would add a breakpoint for it by running a command like this:</p>
<pre>stop in com.example.namespace.ExampleClass.exampleMethod</pre>
<p>Then run the <code>run</code> command. The JRuby and Java code will run until the specified breakpoint is hit, at which point the debugger will stop and let you interact with it again. Here are some more common commands I use:</p>
<ul>
<li><code>where</code> prints a full stack trace. These tend to be very long because of the internals of JRuby but if you scroll to the top you can see what part of the Java code you are in.</li>
<li><code>step</code> advances to the next line of code, stepping into method calls if they are present.</li>
<li><code>next</code> advances to the next line of code, skipping any method calls that might be present.</li>
<li><code>print EXPR</code> evaluates the Java expression EXPR and prints the result. For example, if there is a local variable named "foo" you can type <code>print foo</code> to see its value.</li>
</ul>
This works in both the normal Command Prompt that comes with Windows and also the Git Bash shell that comes with <a href="http://www.git-scm.com/">Git</a>.
There are a couple more things that would be nice to have:
<ul>
<li>It would be nice to have a JDB command that prints out the current filename, method name, and line number. You get this info whenever you run <code>next</code> or <code>step</code>, but sometimes that output ends up really far up in my buffer and I would like a way to quickly see that information again.</li>
<li>It would be nice to do this inside some kind of an graphical interface or IDE. Has anyone gotten that working?</li>
</ul>
By the way, I think JDB is only for debugging Java code. If you want to debug JRuby code there are other tools for that. One that worked for me is the ruby-debug gem, even though people claim it is not maintained.
DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-14394064606230331382013-04-21T16:44:00.001-07:002013-04-21T16:44:41.793-07:00Unusable UI caused by the Windows Forms Anchor property and window size restrictions<p>Suppose you make a Windows desktop application using <a href="http://www.microsoft.com/visualstudio/eng">Visual Studio 2012 Express for Desktop</a> and you want to have a resizable window with a textbox that takes up most of the space and a button at the bottom of the window, as shown below:</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130421_winforms_anchor/desired.png" alt=""/>
<p>There are several ways to do this, but if you are not careful then a flaw in Windows Forms will cause some of your users to see an <i>unusable UI</i> where the button is missing, as shown below:</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130421_winforms_anchor/bad_125dpi.PNG" alt=""/>
<p>In this post I will document the flaw in Windows Forms and why it happens.</p>
<p>Here are the steps for reproducing the flaw:</p>
<ol>
<li>In the Control Panel, change the <b>DPI</b> setting to <b>100%</b>. You can find this under <b>Control Panel > Appearance and Personalization > Display > Change the size of all items</b>, but you can get there faster if you just press the Windows key, type "dpi", and then do the proper keystrokes after that. If you are in Windows 8, you will need to select "Settings" after typing "dpi".</li>
<li>In <b>Visual Studio 2012 Express for Desktop</b>, create a new <b>Windows Forms Application</b>. I used Visual C# but I expect the problem to happen in Visual C++ and Visual Basic too.</li>
<li>Create a <a href="http://msdn.microsoft.com/en-us/library/system.windows.forms.button.aspx">Button</a> and position it in the lower-right corner of the form. Set the <a href="http://msdn.microsoft.com/en-us/library/system.windows.forms.control.anchor.aspx">Anchor</a> property of the button so that it anchors to the bottom and the right. This means that when the Window is resized, the button will move in order to keep two distances constant: the distance between the button's right edge and the window's right edge, and the distance between the button's bottom edge and the window's bottom edge.</li>
<li><i>Optional:</i> Create a <a href="http://msdn.microsoft.com/en-us/library/system.windows.forms.textbox.aspx">TextBox</a> that takes up the rest of the space on the forum. You will need to set its <b>Multiline</b> property to true, and set its <b>Anchor</b> property so that it anchors on all four sides. It will resize with the window.</li>
<li>Resize the form so that it the form's height is about 100 pixels less than the height of your screen.</li>
<li>Run the project and verify that everything behaves as expected. The entire textbox and button should be visible no matter how you resize the window.</li>
<li>In the Control Panel again, change the <b>DPI</b> setting to <b>125%</b>. You will have to log off for the change to take affect. You should notice that everything on your desktop appears bigger than it was before.</li>
<li>Run the program that you built earlier. You should probably just find the executable that was built and run that directly instead of starting Visual Studio. The button will be missing.</li>
</ol>
<p>So what is happening here? There are three features of .NET that are trying to adjust your window:</p>
<ol>
<li><b>Anchoring:</b> The anchoring you set for you controls is trying to make certain distances constant. This is probably done in a SizeChanged event handler or something on the form.</li>
<li><b>Auto-scaling:</b> At startup, when <code>this.ResumeLayout(false);</code> is called in the <code>InitializeComponent</code> method, the <a href="http://msdn.microsoft.com/en-us/library/ms229605.aspx">Auto-scaling</a> feature detects that the form was designed on a system with a different font size than the system on which the program is now running. This will resize the form and the things inside it.</li>
<li><b>Window size limiting:</b> Windows Forms does not let you have a window that is taller or wider than the user's screen.</li>
</ol>
<p>It seems to me like the anchoring and auto-scaling work well together. At startup, the auto-scaling increases your window size to account for the larger font on the user's computer, and expands the window and controls appropriately. However, when the window size limiting procedure runs, it notices that your window is taller than the screen, so it changes the height of the window. Unfortunately, something must be wrong because the anchoring rules do not get applied when the height is decreased. This leaves your button and part of the text box invisible because they are below the bottom edge of the form. This all happens when your application starts up and resizing the window afterward does not help.</p>
<p>This really seems like a bug in the .NET Framework's Windows Forms, and I hope that Microsoft will fix it in the next version. I cannot think of a reason why they would want the anchoring rules to not be applied after the window size limitations are applied.</p>
<p>I tested this at 150% DPI and the problem did not happen.</p>
<h2>Summary of the flaw</h2>
<p>At startup, if your Windows form gets expanded by <a href="http://msdn.microsoft.com/en-us/library/ms229605.aspx">auto-scaling</a> to size larger than the user's screen, it will get resized to fit on the screen. Unfortunately, the <a href="http://msdn.microsoft.com/en-us/library/system.windows.forms.control.anchor.aspx">anchoring</a> rules are not applied after that last resize. If your window's height got decreased, then any elements that were anchored directly to the bottom of the form will have incorrect coordinates. If your window's width got decreased, than any elements anchored directly to the right side of the form will have incorrect coordinates.</p>
<h2>Workarounds</h2>
<p><b>Smaller form:</b> The simplest workaround is to make your form smaller. Let's suppose that the smallest monitor size you want to support is 1024x768. If we assume that the largest DPI setting where this problem occurs is 125%, then we can divide those dimensions by 1.25 to get 819x614. Your form needs to be smaller than 819x614 when displayed at normal DPI. Note that this is just the initial, default size of the form; the user can resize it after the application has started if they want to.</p>
<p><b>Avoid anchoring elements directly to the form:</b> One workaround is to make a <a href="http://msdn.microsoft.com/en-us/library/system.windows.forms.panel.aspx">Panel</a> and set its <b>Dock</b> property to <b>Fill</b>. Put all your elements inside the panel instead of directly on the form. I tested this and it worked. I think that the docking must be done in a better way than anchoring, so the panel is able to receive some event about its size getting reduced, and correctly apply the anchoring rules to its children.</p>
<p>You can download the Visual Studio projects I used to investigate this here: <a href="http://www.davidegrayson.com/for_blog/entry/20130421_winforms_anchor/anchoring.zip">anchoring.zip</a>. AnchorTest is the one with the bug and AnchorFix is the improved one with a docked panel.</p>
<h2>Other flaws related to anchoring</h2>
<p>I suspect that there are more problems with anchoring. Here is a list of the ones I have heard of. Are there any more?</p>
<ul>
<li>I have heard that anchoring does not work correctly with Visual Inheritance, which means having a Form or Control that is a subclass of another. See <a href="http://weblogs.asp.net/rweigelt/archive/2003/09/24/28984.aspx">Roland Weigelt's blog post from 2003</a>.</li>
</ul>
<h2>Conclusion</h2>
This is a pretty rare problem that is unlikely to affect many users. Most people that have small monitor resolutions should really just use the default DPI setting of 100%. Most people who use a non-standard DPI setting should have enough resolution so that the window size limiting does not happen at startup. If you cannot wait for Microsoft to fix this, there are some workarounds you can use.DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com22tag:blogger.com,1999:blog-679854510689771184.post-81190024462850302442013-04-20T17:46:00.000-07:002013-06-04T17:28:40.268-07:00Git and PuTTY in Windows<p><a href="http://git-scm.com/">Git</a> is a very useful tool for keeping track of different versions of files as you make changes to them. In this post, I will talk about how I prefer to install Git in Windows, and how Git is integrated with <a href="http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html">PuTTY</a>, my SSH-client.</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/setup1-path.png" alt=""/>
<p>First of all, I like to install git in <b>C:\git</b> instead of in "Program Files" so that the path has no spaces. Git comes with lots of Unix utilities such as <i>diff</i> and <i>grep</i>. Lots of Unix utilities were developed in an environment where it was highly unusual for a path to have spaces in it. As a result, when these utilities are ported to Windows, some of them have trouble when they encounter spaces in a path. I am not saying that the utilities included with Git necessarily have this problem, but maybe some other Unix utilities on your computer will get screwed up by the presence of the Git Unix utilities on the path. This is not just an academic concern: it actually happened to me and caused problems for our customers. See <a href="http://savannah.gnu.org/patch/?7522">this bug report I filed to the GNU <i>make</i> maintainers</a>. On the other hand, you should install Git in the default location if you plan on redistributing any Unix utilities to your customers; it's better that you should be the first person to experience the bugs in those utilities, rather than your customers.</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/setup2-components.png" alt=""/>
<p>I mostly just installed the default components of git, except that I chose the fancy git-cheetah plugin instead of the simple menus. This means that when I right-click on a folder or file in Windows, I will see some git commands available in the menu, and there will be different commands depending on whether I am in a git repository or not.</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/setup3-path.png" alt=""/>
<p>Git gives you three options for how it will adjust your PATH environment variable. The PATH is a list of directories that contain executables you might want to run from a command line. I chose the third option, which is the most invasive, because I occasionally need to use the plain old Command Prompt (instead of Git Bash), and when I do that I want to have access to useful utilities like <i>grep</i> and GNU <i>find</i>. I have not tested this option for very long though, so if you want to be careful you should pick the middle option. (Just because the middle option mentions cygwin it doesn't mean you have to use cygwin.)</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/setup4-crlf.png" alt=""/>
<p>I think the options on the line-ending conversion setup screen above are pretty self-explanatory. You just need to understand that there is this long history of Windows applications using two bytes (0x0D, 0x0A) to indicate a new line in ASCII text, while Unix applications use just one byte (0x0A). I always chose the first option and it has not caused me any problems.</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/setup5-plink.png" alt=""/>
<p>Git can establish secure connections to remote git repositories using SSH. The git client thankfully does not implement all the details of SSH key management and profile configuration; instead it lets you choose another program to do that work. The default option is to use the version of OpenSSH (ssh.exe) that comes with Git. I choose the other option, which is to use PLink. I will talk more about this below.</p>
<p>After installing Git, you should configure what editor you want to use for commit messages. I usually specify commit messages on the command line with the <code>-m</code> option, but sometimes I just need to use an editor. For example, if you want to write a very long commit message or you want to amend a commit without changing the message, the editor is pretty useful. I like <a href="http://notepad-plus-plus.org/">Notepad++</a>, so I just run the following command to configure git to use Notepad++. The <code>-alwaysOnTop</code> option is used so that I do not get too distracted in the middle of writing my commit message!</p>
<pre class="code">git config --global core.editor "'C:/Program Files (x86)/Notepad++/notepad++.exe' -multiInst -notabbar -nosession -noPlugin -alwaysOnTop"</pre>
<h2>Exploring Git Gui</h2>
<p>If you are going to spend a lot of time with git, I recommend using Git Bash and being familiar with the command-line options for Git. It will help in the long run. However, you might also find it useful to use the included GUI. Here is the context menu you get when you right-click on a file or folder in a git repository, thanks to the git-cheetah plugin:</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/git-cheetah-menu.png" alt=""/>
<p>The only difference between the <i>Git Commit Tool</i> and the <i>Git Gui</i> is that the commit tool closes after you make a commit, and it has a lot fewer features because it just for making commits. Besides the context menu, you can also access either tool from the command line by typing <code>git gui</code> or <code>git citool</code>. This is what the <i>Git Gui</i> looks like:</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/git-gui.png" alt="[picture]"/>
<p>Here is what the <i>Git History</i> screen looks like. I did some editing so you can see three of the right-click context menus that are available on this screen:</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/git-history.png" alt="[picture]"/>
<h2>Git Gui versus TortoiseGit</h2>
<p>There are many options for <a href="http://git-scm.com/downloads/guis">GUIs to use with git</a>. I have used <a href="https://code.google.com/p/tortoisegit/">TortoiseGit</a> (and Tortoise SVN before that) for many years. TortoiseGit does have a very nice interface for showing the history of your git repository, but I find that interface to be buggy and slow at times. If you change one of the checkboxes at the bottom while the window is open, is it not uncommon for that operation to take 5-15 seconds. When that operation is done, it is not uncommon for the lines drawn between the commits to be incorrect.</p>
<p>TortoiseGit does have a nice shell extension for Windows that puts icon overlays on your files to indicate which ones have changed in git. However, I cannot trust those icons anymore because they have too often been incorrect in my experience. For example, I will see that some folder has a red icon indicating that it has changed, but then I run "git diff" and see that in fact nothing has changed. On top of that, I have occasionally experienced even weirder problems in Windows Explorer that I suspect might be the fault of TortoiseGit.</p>
<p>TortoiseGit and Git Gui have a different approach to UI design. In TortoiseGit, the center of your experience will be a really long sub-menu of right-click context menu. It contains about 25 different operations you might want to perform. Actually, I move the most-common operations up one level in the menu structure so I don't have to go into that large menu too often. With Git Gui, the center of your experience will be the Git Gui window.</p>
<p>I have barely used Git Gui at all, but my impression is that it is faster and less buggy than TortoiseGit. Once I get used to it, I think my productivity will increase.</p>
<p>Side note: One downside of <i>Git Gui</i> is that the people who named it used the incorrect capitalization of the acronym <a href="http://en.wikipedia.org/wiki/Graphical_user_interface">GUI</a>. When you write a blog post about <i>Git Gui</i> you might look dumb no matter which capitalization you choose to use.</p>
<h2>How Git works with PuTTY</h2>
<p>I find it interesting to explore how git uses PuTTY to take care of your SSH connections. Also, you might need this information some day if you are debugging your git SSH connection. Here is the block diagram of what happens when you type "git fetch":</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/git-fetch-diagram.png" alt=""/>
<p>The boxes represent executable processes that are running during the fetch operation, and the arrows represent the flow of information. When you type <code>git fetch</code> in bash or some other shell, your shell will spawn a new git process and let you see the characters it outputs on the standard output. The git process will check the <code>$GIT_SSH</code> environment variable, which tells it which SSH client to use. In my case, I want <code>$GIT_SSH</code> to be "C:\Program Files (x86)\PuTTY\plink.exe" so that git uses the plink.exe which comes with PuTTY. Git spawns a new plink process with two arguments. The first argument is the remote host name or PuTTY session name. The second argument is a command to run on the remote machine. Git will take over the standard input and standard output of the plink process, but the error output from plink is usefully redirected to your shell. After plink establishes a connection to the remote machine, the SSH daemon (sshd) on the remote machine will run that command. (Actually, it might run it indirectly through a shell like bash.) The command will be something like "git-upload-pack '/path/to/repo'". This starts a new git process on the remote machine and tells it the path of the repository to look at. At this point there is a git process running on your machine and one running on the remote machine. They coordinate in some smart way and figure out what data needs to be fetched to your machine.</p>
<p>Here is a fun shell session that I used to figure out this stuff. The name "davidegrayson" is the name of a PuTTY session that establishes an SSH connection to my web server. The directory '/home/public/' holds a git repo for my website. In the last command, you can see a little sampling of the protocol that the two git processes use to talk to eachother.</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20130420_git_win8/git-fetch-hack.png" alt=""/>
This was kind of a random post but I hope you will find it useful! At least I will find it useful the next time I am installing Git in Windows!
DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1tag:blogger.com,1999:blog-679854510689771184.post-17747040034106045892013-01-13T13:34:00.001-08:002015-03-04T19:41:58.104-08:00Ruby 2.0: Helpers should be refinements<p>In Ruby, people sometimes make modules called "helpers" that define some useful methods:</p>
<pre><code class="language-ruby">
module MyHelper
def add(a,b)
a + b
end
end
</code></pre>
<p>If you include the module in a class, you can call the helper methods easily without any kind of prefix:</p>
<pre><code class="language-ruby">
class MyClass
include MyHelper
def join(str1, str2)
add str1, str2
end
end
puts MyClass.new.join("a", "b") # => "ab"
</code></pre>
<p>However, I think this has bad semantics. Look at the output below and think about why it is bad:</p>
<pre><code class="language-ruby">
puts MyClass.new.is_a?(MyHelper) # => true
puts MyClass.new.add(4, 3) # => 7
</code></pre>
<p>The fact that <code>MyClass</code> uses methods from <code>MyHelper</code> should be an internal implementation detail, but that information is leaking out to the users of <code>MyClass</code> in all kinds of ways.</p>
<p>The Ruby <code>is_a?</code> function returns true, telling you that an instance of <code>MyClass</code> <em>is</em> a <code>MyHelper</code>. However, if you design a structure like this, it is more likely that your mental model is that an instance <em>uses</em> the helper to accomplish its tasks without <em>being</em> a helper.</p>
<p>There are two more practical problems with this setup. First, people are free to call the helper method <code>#add</code> on instances of MyClass. That does not make any sense, so we should design our code to disallow that. Second, the <code>#add</code> method would show up in the list of methods returned by <code>MyClass.new.methods</code>; calling <code>#methods</code> in IRB is a super useful way to look for features of an unfamiliar class, and our setup makes that harder because <code>#methods</code> will return lots of junk. An easy way to fix these two problems is to add a new line that says <code>private</code> at the top of the module, but I think that is not as good as the solution I propose below.</p>
<h2>Refinements</h2>
<p>In Ruby 2.0, the language was expanded to include <b>refinements</b>. If you have not heard of refinements, <a href="http://lmgtfy.com/?q=refinements+ruby">Google it</a> now. However, beware that <code>Module#using</code> was recently <a href="https://github.com/ruby/ruby/commit/537297d1cbcd2ed97488774e67c4fc001282a658">removed</a> so nearly all of the examples you will find are wrong. You can mostly only call <code>using</code> from the top level of a Ruby file.</p>
<p>Refinements were mainly intended to make monkey-patching safer, but I believe they can also greatly improve the way we use helpers. The <code>Object</code> class is the parent class of (almost) every Ruby object, so if we refine it with some helper methods, those methods will be available on every object, including <code>self</code>. This means we can call them easily without any kind of prefix:</p>
<pre><code class="language-ruby">
module MyHelper
refine Object do
def add(a, b)
a + b
end
end
end
using MyHelper
class MyClass
def join(str1, str2)
add str1, str2
end
end
puts MyClass.new.join("f", "g") # => "fg"
puts MyClass.new.is_a?(MyHelper) # => false
puts MyClass.new.respond_to?(:add) # => false
puts MyClass.new.add(4, 5) # throws NoMethodError if line
# is in a different file
</code></pre>
<p>As you can see above, by making a few easy changes to the helper and the code that uses it, we are able to fix all of the semantic problems we were having earlier. This is nice, except for one caveat which I will get to later.</p>
<h2>Changing a helper module into a refinement</h2>
<p>We had to add two lines of code to <code>MyHelper</code> to use it as a refinement. What if you do not want to edit the original module? Maybe the module is part of a third-party library or maybe you are not ready to refactor all the code that uses the module. The following works:</p>
<pre><code class="language-ruby">
module MyRefinement
refine Object do
include MyHelper
end
end
</code></pre>
<p>Generally, including a module with <code>include</code> is equivalent to defining some functions in the same place, and that continues to be true with refinements. This makes me happy.</p>
<h2>Making it even better, maybe</h2>
<p>Refining <code>Object</code> is troublesome because the code that uses the helper would be allowed to call helper methods on random objects. Again, this does not make sense so our code should disallow it:</p>
<pre><code class="language-ruby">
44.add(4, 5) # => 9
</code></pre>
<p>Maybe this is OK if you are the only one writing code that uses your helpers, but if you are writing a system or framework that involves helpers, it would be so much nicer to just refine the class that actually uses the helpers. We can do that, as shown below, but it gets a little messy because of the limitations of the Ruby language:</p>
<pre><code class="language-ruby">
# This method is defined once per project.
def helper(mod, in_class: Object)
Module.new do
refine in_class do
include mod
end
end
end
# This is a helper module.
module MyHelper
def add(a,b)
a + b
end
end
# This is how you write code to use the helper module:
class MyClass; end
using helper MyHelper, in_class: MyClass
class MyClass
def join(str1, str2)
add str1, str2
end
end
puts MyClass.new.join("a", "b") # => "ab"
</code></pre>
<p>Unfortunately, I think this is the best we can do if we don't want to refine the <code>Object</code> class. If anyone—especially Ruby language designers—has an idea of how to clean this up, let me know! I will put all of my <a href="https://gist.github.com/4526439">failed ideas here</a>.</p>
<p><b>I think we are very close to having a great new use for refinements, but maybe we need a new feature or two in the language to make it feasible.</b></p>
<p>Note: refinements are experimental, and the behavior may change in future versions of Ruby! This article was based on <b>ruby 2.0.0dev (2013-01-07 trunk 38733) [x86_64-linux]</b>, also known as <b>ruby 2.0.0-rc1</b>.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1tag:blogger.com,1999:blog-679854510689771184.post-45170829189138306602013-01-07T22:13:00.004-08:002013-01-07T22:13:48.556-08:00Practical Windows Code and Driver Signing<p>I recently wrote a 7000-word document that should contain everything most developers need to know about Microsoft Windows code and driver signing. You can find it on my website: <a href="http://www.davidegrayson.com/signing/">Practical Windows Code and Driver Signing</a>.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-54358997583305149752012-12-19T23:43:00.002-08:002012-12-19T23:43:30.883-08:00Fresh & Easy receipt printer test<p>Look what I found hanging out of the receipt printer at the self-service checkout at <a href="http://www.freshandeasy.com/">Fresh & Easy</a> (grocery store) the other day:</p>
<img src="http://www.davidegrayson.com/for_blog/entry/20121219_fe_receipt/fresh_and_easy_printer_test.jpg"/>
<p>Some of the screwiness of this image is due to my handheld scanner. The actual receipt was not printed crookedly.</p>
<p>Here's a transcript so people can find this by searching on Google:</p>
<pre>
--------------------------------------------
NCR 734x-F307 Printer: Thermal Receipt
NCR 71xx Printer/MICR/CD/ChkScan Service
Object
OPOS Spec Compliancy Version: 1.7.000
--------------------------------------------
OPOS POSPrinter Control 1.10.001 [Public,
by CRM/RCS-Dayton]
OPOS Spec Compliancy Version: 1.10.001
--------------------------------------------
NCR Printer Service Object
NCR Release File Version: 2.5.1.50
--------------------------------------------
SH/SW: 0123456789:;<=>?@ABCDEFGHIJKLMNOPQRST
DH/SW: 0123456789:;<=>?@ABCDEFGHIJKLMNOPQRST
SH/DW: 0123456789:;<=>
DH/DW: 0123456789:;<=>
--------------------------------------------
CharacterSet-437
(List-437,737,850,852,858,860,862,863,864,86
5,866,874,932,1252,101); LineChars-44
(List-44,56)
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK
LMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
xyz{|}~
</pre>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com16tag:blogger.com,1999:blog-679854510689771184.post-77042492183042992462012-12-03T22:20:00.001-08:002013-04-20T17:58:02.194-07:00Getting git 1.8.0 to work with PuTTY/plink/pageant 0.62<p>I am successfully running the following things together:</p>
<ul>
<li>Windows 8</li>
<li>Git-1.8.0-preview20121022.exe from <a href="http://code.google.com/p/msysgit/downloads/list">here</a></li>
<li>PuTTY, plink, and pageant, all version 0.62, from <a href="http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html">here</a></li>
</ul>
<p>
I'm not sure if the Startup menu exists any more in Windows 8. To make pageant.exe run at startup and load my key, I used regedit to add a registry value under <b>HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run</b>. The value is a String, the name can be whatever, and the value should be something like:</p>
<pre>"C:\Program Files (x86)\PuTTY\pageant.exe" "C:\your\private\key.ppk"</pre>
<p>
By default, the version of SSH that ships with msysgit does not try to access pageant. You have to configure git to use plink.exe instead. Lots of people on the internet will <a href="https://groups.google.com/forum/?fromgroups=#!topic/msysgit/NSjkX7kUDJM">tell</a> you that you must set the GIT_SSH environment variable to be the full path to plink.exe, but for some reason that wasn't working for me this time. I'm sure this is not optimal, but my solution was to move plink.exe to "C:\Program Files (x86)\Git\bin\ssh.exe", overwriting the original ssh.exe. (I saved a copy of the of the original ssh.exe just in case I need it later.)</p>
<p>
Anyway, it works now so I can continue working on all of <a href="https://github.com/DavidEGrayson">my favorite coding projects</a>.
</p>
<p>
<b>UPDATE:</b> See my new and improved article entitled <a href="http://blog.davidegrayson.com/2013/04/git-and-putty-in-windows.html">Git and PuTTY in Windows</a>.</p>DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com2tag:blogger.com,1999:blog-679854510689771184.post-83342855923001318822012-11-26T21:56:00.001-08:002014-07-11T10:12:07.871-07:00Orientation sensing with the Raspberry Pi and MinIMU-9 v2
<p>The <a href="http://www.raspberrypi.org/">Raspberry Pi</a> is an ideal platform for experimenting with inertial measurement units (IMUs). These IMUs are usually I²C devices, so it is straightforward to access them using the Raspberry Pi's I²C bus. The integrated 3D graphics on the Raspberry Pi are useful for visualizing the results.</p>
<p>I have made some software that allows you to connect the <a href="http://www.pololu.com/catalog/product/1268" title="Pololu - MinIMU-9 v2 Gyro, Accelerometer, and Compass (L3GD20 and LSM303DLHC Carrier)">MinIMU-9 v2</a> from <a href="http://www.pololu.com/">Pololu Robotics & Electronics</a> to your Raspberry Pi and calculate the orientation of the IMU. I also wrote a visualization program that shows the results in 3D. Check out the demo in this video:</p>
<iframe width="560" height="315" src="http://www.youtube.com/embed/8daR-wtQIx0" frameborder="0" allowfullscreen></iframe>
<p>The software is open source and I have written a <a href="https://github.com/DavidEGrayson/minimu9-ahrs/wiki">tutorial</a> for it. The only items needed besides your Raspberry Pi setup are:</p>
<ul>
<li>A <a href="http://www.pololu.com/catalog/product/1268" title="Pololu - MinIMU-9 v2 Gyro, Accelerometer, and Compass (L3GD20 and LSM303DLHC Carrier)">MinIMU-9 v2</a> from Pololu, or the older MinIMU-9.
<li>Four <a href="http://www.pololu.com/catalog/category/66">female-female jumper wires</a>.</li>
</ul>
<p>Soldering is required: the male header pins that come with the MinIMU-9 v2 need to be soldered in.</p>
<p>The MinIMU-9 v2 can be conveniently powered from the Raspberry Pi's 3V3 power line. The photos below show how to connect a MinIMU-9 v2 to a Raspberry Pi:</p>
<a target="_new" href="http://www.davidegrayson.com/minimu9-ahrs/wiring.jpg">
<img alt="Wiring" src="http://www.davidegrayson.com/minimu9-ahrs/wiring_560px.jpg" />
</a>
<p>The software consists mainly of two programs, both written in C++. The <a href="https://github.com/DavidEGrayson/minimu9-ahrs">minimu9-ahrs</a> program reads data from the MinIMU-9's sensors via I²C, calculates the orientation of the IMU, and outputs data to the standard output. Internally, it represents the orientation with a quaternion, but several different output options are available: direction cosine matrix, quaternion, euler angles, and raw sensor data. The <a href="https://github.com/DavidEGrayson/ahrs-visualizer">ahrs-visualizer</a> program reads direction cosine matrix data from its standard input and displays a 3D representation of the MinIMU-9 v2 using OpenGL. I also used <a href="http://packages.debian.org/search?keywords=libi2c-dev">libi2c</a>, <a href="http://www.libpng.org/pub/png/libpng.html">libpng</a>, the <a href="http://www.boost.org/libs/program_options/">Boost program_options library</a>, and <a href="http://eigen.tuxfamily.org/">Eigen</a>. There is a Python script that helps calibrate the magnetometer. For easy installation of the software, I have created Debian packages.</p>
<p>For more information, see the <a href="https://github.com/DavidEGrayson/minimu9-ahrs/wiki">tutorial</a>! I would love to see what people build with this, so please post comments below!</p>
DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com87tag:blogger.com,1999:blog-679854510689771184.post-71564493199097229692012-11-26T14:44:00.002-08:002012-11-26T14:44:41.864-08:00PIC18F14K50 errata: I was right all alongIn March 2009 (3.5 years ago), <a href="http://www.microchip.com/forums/tm.aspx?m=411835">I reported a problem with the PIC18F14K50</a> on the Microchip forum. This month, they finally added it to the <a href="http://ww1.microchip.com/downloads/en/DeviceDoc/80411H.pdf">errata list</a>.
This should somehow give me more credibility the next time I report a bug to a silicon manufacturer!
DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-51124361132670444362012-07-10T22:17:00.000-07:002015-03-04T19:39:22.363-08:00Atmel Studio bug caused by --Wl,--gc-sectionsI have found a bug in <a href="http://www.atmel.com/microsite/atmel_studio6/default.aspx">Atmel Studio 6.0.1843</a> (with AVR Toolchain 8 Bit 3.4.663 - GCC 4.6.2) that results in RAM variables being placed starting at address 0x60 instead of 0x100 if the <code>--Wl,--gc-sections</code> flag is enabled and the <code>.data</code> section is empty. This makes the RAM variables overlap with memory-mapped <a href="http://en.wikipedia.org/wiki/Special_function_register">SFR</a>s, so they do not function properly.
<br/><br/>
You can follow these steps to reproduce the bug:
<ol>
<li>Make a new project with the GCC C Executable Project template and select the ATmega328P. (I believe this bug affects other parts too.)</li>
<li>In the project properties, check the "Garbage collect unused sections (-Wl,--gc-sections)" checkbox.</li>
<li>Write the following in main.c:
<pre><code class="language-c">
#include <avr/io.h>
unsigned char foo = 0;
int main(void)
{
while(1){ PORTC = foo++; }
}
</code></pre>
</li>
<li>Build the project.</li>
</ol>
Now look at the .map output file. You will see the following line, which indicates that foo is at address 0x60.
<pre>
0x00800060 foo
</pre>
Unfortunately, address 0x60 is the location of WDTCSR, so putting a variable there does not work well.
<br/><br/>
The problem goes away if you ensure that the <code>.data</code> section is non-empty by making a static variable that is initialized to a non-zero value:
<pre class="c" name="code">
static void __attribute__((used)) workaround(void)
{
static volatile unsigned char w = 1;
w++;
}
</pre>
I believe that Atmel Studio is affected by <a href="http://sourceware.org/bugzilla/show_bug.cgi?id=13697">bug #13697</a> from GNU binutils.
<br/><br/>
I hope that the next version of Atmel Studio comes with a patched or updated binutils that does not suffer from this problem. I have reported the bug to Atmel.
<br/><br/>
<b><i>Update 2012-07-18:</i></b> A member of the Atmel AVR Toolchain Team has responded to my bug report. He said: "Thanks. We will have this fix available with our next release."DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com0tag:blogger.com,1999:blog-679854510689771184.post-86277545773468149012012-06-20T23:07:00.001-07:002012-06-21T10:45:05.807-07:00Using I²C on the Raspberry PiLast night I succeeded in getting my <a href="http://www.raspberrypi.org/">Raspberry Pi</a> to do some basic <a href="http://en.wikipedia.org/wiki/I%C2%B2C">I²C</a> (also known as I2C, TWI, or SMBus) communication. I had to look around in a couple different places on the web to figure out how to do it, so I thought I would make this blog post to consolidate the knowledge.<br />
<br />
The first step, of course, is to set up an SD card suitable for the Raspberry Pi. I used the official Debian image from the Raspberry Pi Foundation.<br />
<br />
Unfortunately, this official image has no support for I²C! You need to enter <a href="http://www.bootc.net/projects/raspberry-pi-kernel/">Chris's Digital Realm</a> for that. Follow the instructions on Chris Boot's site to upgrade your Raspberry Pi firmware and install his Raspberry Pi kernel, which has drivers for I²C. Now if you run <code>lsmod</code> you should see a lot of good stuff, such as the <b>i2c_bcm2708</b> module.<br />
<br />
Once you are running the new kernel, you will need to run <code>sudo modprobe i2c_dev</code>. You should also add <code>i2c_dev</code> to the <code>/etc/modules.conf</code> file so that it loads automatically the next time you boot up the Raspberry Pi. Now you should see two I2C devices:<br />
<pre>pi@raspberrypi:~/$ ls -l /dev/i2c*
crw------- 1 root root 89, 0 Dec 31 1969 /dev/i2c-0
crw------- 1 root root 89, 1 Dec 31 1969 /dev/i2c-1
</pre><br />
You will need to connect your device to the Raspberry Pi. I used Pololu <a href="http://www.pololu.com/catalog/category/66">Female-Female Premium Jumper Wires</a> to make the following connections between the <a href="http://elinux.org/File:GPIOs.png">Raspberry Pi's GPIO header</a> and my device:<br />
<ul><li>Raspberry Pi GND to device GND</li>
<li>Raspberry Pi SCL to device SCL</li>
<li>Raspberry Pi SDA to device SDA</li>
<li>Raspberry Pi 3V3 Power to device's power input. This line can only provide a few tens of milliamps so you should look up how much current your device draws and make sure you can power it at 3.3 V. The Raspberry Pi's 5V power line is also available.<br />
</ul>At this point, you can use simple C system calls to communicate with I2C devices. Here is some code I wrote to read the "Who Am I" register from the LSM303DLM on a Pololu <a href="http://www.pololu.com/catalog/product/1265">MinIMU-9 Gyro, Accelerometer, and Compass (L3G4200D and LSM303DLM Carrier)</a>: <pre class="c" name="code">#include <unistd.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <linux/i2c-dev.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#define MAG_ADDRESS (0x3C >> 1)
#define LSM303_WHO_AM_I_M (0x0F)
int main()
{
char buf[10];
const char * devName = "/dev/i2c-0";
// Open up the I2C bus
int file = open(devName, O_RDWR);
if (file == -1)
{
perror(devName);
exit(1);
}
// Specify the address of the slave device.
if (ioctl(file, I2C_SLAVE, MAG_ADDRESS) < 0)
{
perror("Failed to acquire bus access and/or talk to slave");
exit(1);
}
// Write a byte to the slave.
buf[0] = LSM303_WHO_AM_I_M;
if (write(file, buf, 1) != 1)
{
perror("Failed to write to the i2c bus");
exit(1);
}
// Read a byte from the slave.
if (read(file,buf,1) != 1)
{
perror("Failed to read from the i2c bus");
exit(1);
}
printf("result: 0x%02X\n", buf[0]);
return 0;
}
</pre>When I ran this code, it printed 0x3C which means that it successfully talked to the device! There might be a simpler way to read the register; see the <a href="http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/i2c/dev-interface">Linux Kernel I2C Documentation</a>.<br />
<br />
In conclusion, it was not too hard to get I²C working on the Raspberry Pi. The biggest hurdle is that the official Debian image/kernel has no support for it. This is kind of surprising because the Raspberry Pi team purposefully brought the I²C lines out to their GPIO header and the product has been selling for several months now.<br />DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com21tag:blogger.com,1999:blog-679854510689771184.post-78667371708928441822012-03-21T22:27:00.007-07:002013-04-17T08:30:13.067-07:00The Ruby Enumerable moduleWhen programming in the Ruby language, it is essential to know about the Enumerable module. The more Enumerable methods you know, the more concise and elegant your code will be. The Enumerable module is mixed into many common classes such as Array, Hash, and Range. It is also mixed into some not-so-common classes such as Set, Struct, and Enumerator.<br />
<br />
The <a href="http://www.ruby-doc.org/core-1.9.3/Enumerable.html">official documentation of Ruby 1.9.3's Enumerable module</a> is good, but I don't like the alphabetical organization. I think you can understand what's going on better if you group the methods into similar categories. To that end, I have created a <a href="http://www.davidegrayson.com/ruby/enumerable/enumerable_methods.pdf">Ruby 1.9.3 Enumerable module quick reference</a>. It's a one-page sheet that briefly documents ALL of enumerable's methods. Check it out!<br />
<br />
Download links:<br />
<br />
<a href="http://www.davidegrayson.com/ruby/enumerable/enumerable_methods.pdf" style="font-weight:bold;">Ruby 1.9.3 Enumerable module quick reference (PDF)</a><br />
<br />
<a href="http://www.davidegrayson.com/ruby/enumerable/enumerable_methods.ods" style="font-weight:bold;">Ruby 1.9.3 Enumerable module quick reference (ODS)</a><br />
<br />
Let me know if there are any errors or you think it could be better in some way!DavidEGraysonhttp://www.blogger.com/profile/14463390566152982599noreply@blogger.com1