Tuesday, October 9, 2012

How to make a .NET (Excel) Add-In Using Visual Basic Express 2010

Automation add-ins are easier than what follows and are the way to go if you just want a few custom functions.  Actually, for efficiency's sake, you might want to just use VBA functions added to Module1  (must be Module1—no renaming to a more suitable name) and then save the workbook as an Excel Add-In (*.xlam).  But, if it is user interaction or in some way making Excel "do stuff" (rather than just return a single result or an array of results), an add-in which implements the IDTExtensibility2 and IRibbonExtensibility interfaces may be a good strategy.  (The main alternative is to use VBA.)

Making an Excel Add-in in VB.NET involves the following (the ordering of the steps is not absolute):
  1. Ensure you have Office installed on the development machine and have the Primary Interop Assemblies (PIAs) installed in the Global Assembly Cache (GAC).  You can download them from Microsoft.  (The Microsoft installer will put the PIAs in the GAC.)
  2. Start a Class Library project in VB.NET.
  3. It's a good idea to generate your own strong name using the sn.exe tool (part of the .NET framework).  You only need one for multiple projects.  In a nutshell, you use the sn.exe tool to create a *.snk file which you will tell VB.NET about.
    • let VB.NET know where your *.snk file is by going to Project/Properties..., under Signing check "Sign the assembly" and choose a strong name key file.  You will be able to browse to where your *.snk file is
    • if your add-in is intended to be widely distributed, and especially if you want to sell your add-in, you would be well-advised to learn more about this topic to ensure your end-users don't get warning messages that cause them to be (perhaps unduly) alarmed.
  4. Apparently, step 4 is unnecessary, but I did step 4, so here it is:  In the project properties, go to Application and click the button Assembly Information....  In this screen, check the Make assembly COM-Visible box.
  5. Create a GUID.  There are several ways but a program called guidgen.exe may already be on your computer which will do the job.  Just search your computer for it.  It is located under C:\Program Files\Microsoft SDKs\Windows\... on my computer.
    • Copy and Paste the GUID into the Assembly Information... dialog box referred to in step 4.
  6. Encapsulate your project in a namespace either by typing it explicitly in the files or by specifying a root namespace in the project properties.
  7. Add references to needed PIAs, and extensibility (.NET tab)
    • It is recommended that you add these by referencing the Interop Assembly on the .NET tab, not the COM tab. You may need to browse for this: Look under C:\Windows\assembly\gac_msil\Microsoft.Office.Interop.Excel\...
  8. Create a class named Connect (seems that the name has to be Connect) which implements the IDTExtensibility2 and IRibbonExtensibility interfaces.
    • Implements Extensibility.IDTExtensibility2, Microsoft.Office.Core.IRibbonExtensibility
  9. Implement the methods of the interfaces.
  10. Decorate the class with COM visibility specifications and include functions for creating and deleting custom registry settings.
  11. Register your assembly with regasm.exe.
Steps 8, 9, and 10 are implemented in the following code snippet, which has been parameterized somewhat.


The functions RegisterFunction and UnregisterFunction are invoked by utilities like regasm.exe which comes with the .NET Framework.  I have created a batch file which I run in order to register the assembly.  You will need to modify it to suit your assembly.



Note that once the Add-in is registered and added, you will not need to reregister it after every Build (while developing).  

Thursday, September 6, 2012

Get Your Text Data Into Maxima

Comma Separated Value (CSV) files are often a convenient way to move your text data between a variety of software products. The makeup of a CSV file is straightforward and can be observed by opening up a CSV file in a text editor. Excel provides the ability to save a worksheet to a CSV file.

Once your data is in a CSV file, you can read it into Maxima—with a little bit of code.

But, first, some house-keeping. In order to have convenient access to routines that you or someone else has written, you'll want to put the files that contain this code on Maxima's search path. The variables file_search_maxima and file_search_lisp, indicate the paths that Maxima will look through to find code to load. You can affect these values by modifying these during start-up (or any other run-time). Place a file called maxima-init.mac in the location indicated by the environment variable MAXIMA_USERDIR and place some code to the effect of the following in it:

file_search_maxima: 
    append(file_search_maxima, 
           ["fullpathto/Useful/Maxima/Functions/$$$.{mac,mc}"]);
file_search_lisp: 
    append(file_search_lisp, 
           ["fullpathto/Useful/Maxima/Functions/$$$.{o,lisp,lsp}"]);

(See here for more settings and see chapter 32. Runtime Environment.)

To see more about how to interoperate between Lisp and Maxima, see chapter 3 of the Maxima manual. The downloadable sample code shows how to use some of these techniques.

Download the following two files and place them in the directory you have added to your search path.
  1. CSV.lisp
  2. CSV.mac
Start up Maxima and then type and execute:

load(csv);

You will now have access to 4 functions to help you get your text data into Maxima.


Function:
CSVread(filename)
CSVread(filename,linesToSkip)

CSVread will read in the data in the file and turn it into a list of lists of string data. Each line of data becomes a separate list. Optionally, you may specify a positive integer value indicating how many rows to discard, which is handy if your data file contains (possibly multiple) header rows.


Function:
CSVNumberColumns(filename, skips, columnsToKeep)

CSVNumberColumns reads data in from the file specified by filename and skips skips rows of data (which may be 0). Additionally, it will excerpt the file by only retaining the columns requested in the list columnsToKeep. To keep all of the columns, pass in an empty list ([]). If you want the first and second column only, pass in [1,2]. All data in the columns which are to be retained is expected to be a valid Maxima expression in string form. Numeric or algebraic expressions qualify. Specifically, the data items must be something which the Maxima function parse_string can interpret.


Function:
MakeMatrix(d)

MakeMatrix creates a matrix with the data d, which is a list of lists. This is handy when you need your data in matrix form to pass along to a least squares function in Maxima, such as lsquares_estimates. Note that this function does not create a "deep" copy of the data in d. It is really just a separate "view" of the same data. As a note, I found that the display of the matrix results would sometimes be transposed. Only seemed to happen in output cells in Maxima and actual data format was as expected anyway (display glitch in my version of Maxima but not a consistent one).



Example:

load(csv)$ /* only needed once per Maxima start-up */
d: CSVNumberColumns("D:/path/circledata.csv",1,[2,3]);
m: MakeMatrix(d);

(%o9) [[1007.265919151515,1000.932075151515],[1008.902086,1002.038759],[1010.969034,1002.554592],[1013.1443,1002.268315],[1015.598881,1000.804232],[
1016.837804,998.9016388],[1017.35833,996.4303561],[1016.548809,993.5742165],[1014.687232,991.6508387],[1012.069706,990.6322456],[1009.539441,990.933599],[
1007.03337,992.4675972],[1005.545131,995.07334],[1005.520783,997.8698301]]
(%o10) matrix([1007.265919151515,1000.932075151515],[1008.902086,1002.038759],[1010.969034,1002.554592],[1013.1443,1002.268315],[1015.598881,1000.804232],[1016.837804,998.9016388],[1017.35833,996.4303561],[1016.548809,993.5742165],[1014.687232,991.6508387],[1012.069706,990.6322456],[1009.539441,990.933599],[1007.03337,992.4675972],[1005.545131,995.07334],[1005.520783,997.8698301])

Local Variables, Function Parameters, and Blocks In Maxima

One of the things that was a real hang-up for me when I started programming in Maxima was its seeming lack of variable scoping rules.  It made some irritating and non-intuitive things happen.  But, because I was always in “get-it-done” mode, I didn’t stop to figure out if there was a better way.  I would resort to adding letters to the name that I wanted to use.  If my functions were going to use a, I would instead name those aa, so that my variables in the “true” global realm wouldn’t be overwritten after the next function call.  My convention would then be, “nice names in the global realm, ugly names in the local realm.”  This is far from ideal.  The fact is, Maxima defaults to global variables, but makes it easy to declare local variables.

Maxima’s primary scoping mechanism is the “block.”  A block starts with block( and ends with ).  Each statement within a block is separated by a comma (,).  The first statement in your block should normally be [v_1, v_2,…,v_n], where v_1, v_2, etc., are variables that you wish to be local (don’t literally type v_1, etc.).  If you don’t want any local variables, then omit the local statement.

What Maxima actually does is not quite the same as in most programming languages, although the apparent runtime behaviour will be the same, if the [v_1, v_2,…,v_n] statement is used as above.  Maxima saves the current values of the variables in the [v_1, v_2,…,v_n] statement and essentially “zeros out” (removes their properties) those variables.  They become blank slates.  When Maxima exits the block() in which the variable was declared as local, its current properties are removed and the saved values/properties are restored.  As far as I know, Maxima does not support multi-threading.  This is probably a good thing, as temporarily preventing a variable from “existing” (as itself, that is; a variable with the same name is not the same variable itself, philosophically) is problematic if that variable could be accessed at any time by a separate thread.

This behaviour is different from most other programming languages which use “dynamic scoping” in that there are no save and restore steps of the kind we are talking about here (in most other languages).  In Maxima, variables have names at run-time and are referenced according to their name.  In most languages, the name of a variable is solely a convenience for programmers.  At run-time, values will move to various parts of the program memory space and the memory addresses will be determined on the fly (e.g., "placed on the stack").  Just because the same name is used in the source code for two variable references, doesn’t mean the compiled code will have references to the same memory location.  For example, t declared in function1() has nothing to do with t declared in function2().  These memory locations will be on the stack and exist only while each function is being executed.  This, again, is what is true in most programming languages, not in Maxima.

When it comes to function parameters, however, Maxima`s behaviour is not unlike most other languages.  That is, the parameters are formal.  They are not considered to constitute a variable declaration.  When I write

f(x) := x^2;

x is not declared as a variable.
Even if I write

RotationalEquivalenceSet(n, bits) := block(
    local(res, size),
    res: {n},
    size: 2^bits,
    if n >= size then return({}),
    n: 2*n,
    if n >= size then n: n - size + 1,
    do (
        if elementp(n, res) then return(res),
        res: union(res, {n}),
        n: 2 * n,
        if  n >= size then n: n - size + 1
    )   
);


n is not declared as a variable.  This is a very helpful thing.  If n was assigned some value before the function call, nothing that I do inside this function call (which uses n as the name of a parameter) affects that value.  The value of n will remain the same after this function has been invoked.  Here is an input/output example of the behaviour:

/* [wxMaxima: input   start ] */
n: x^2;
RotationalEquivalenceSet(3,5);
n;
RotationalEquivalenceSet(5,5);
n;
/* [wxMaxima: input   end   ] */

(%o21) x^2
(%o22) {3,6,12,17,24}
(%o23) x^2
(%o24) {5,9,10,18,20}
(%o25) x^2


Note that I am using := to declare these functions.  You may end up with different (and undesired) behaviour if you don’t use the “delayed evaluation” assignment statement.  If you want the usual programming paradigm with your function declaration, use := as per my sample code above.

Saturday, August 11, 2012

DXFert: An Example VBA Add-In with Custom Ribbon

I implemented this project in Excel 2010. Only the Ribbon modifications are likely to be version dependent (unless you go way, way back). The form and macro in this project should work in any version of Excel.  

To use the macro launchmyForm(control as IRibbonControl)without the Ribbon buttons, such as you would need to do in an earlier version of Excel, you will need to either change

   launchmyForm(control As IRibbonControl) to launchmyForm() 

or 

write another macro that calls launchmyForm(someDummyObject).

In this post, I am going to walk through the building blocks of the project. Except for Excel, everything you need for this project is freely available for download.

What You Will Need to Develop Your Own Similar Project
  1. Excel 2007 or later.
  2. DXF file format documentation from AutoDesk
  3. CustomUI Editor Tool from Open XML Developer
  4. A program to allow you to test your DXF output (if that's what you're going for), such as DWG TrueView.
Overview

You need to 
  1. create a UserForm within a macro-enabled workbook (*.xlsm)
  2. write a one-liner macro that invokes the UserForm
  3. create a Ribbon interface which references the one-liner macro, and, finally,
  4. save the project as an Add-In (*.xlam) and activate it.
Creating the UserForm

If you don't have the developer tab available to you,...why ever not? Turn the silly thing on! To learn about developing user forms in Excel generally, there are lots of examples on the web, but I'll just give you a few key pieces of code that were important in my project.

Here's the code I used to open a file SaveAs dialog to help the user indicate the name and path for the file that the results will be exported:


This should work in very early versions of Excel all the way up to the present (version 2010) and probably future versions as well.  My main beef with it is there is a lack of control.  I would like it to complain about the selection of a file which already exists, but if I really wanted to I could include that complaint in my macro code right before I follow through with the export (which I didn't do—but you can, if you wish).

When you want to terminate the form (either by cancelling or after successful completion), you'll want use something like this:


You can learn a lot about the DXF format by looking at my code in the download.  (Hint:  If you have the Add-In added, you don't need to open up the file separately to see the code.  Just go to the Visual Basic Editor (button on the developer tab) and locate the user form DXFout_Form.  Right-click it and choose View Code.)  See page 143 of the AutoDesk's DXF format documentation for a description of the fields in the polyline Entity.

The one-liner code which the Ribbon will call is pretty easy:


The Ribbon expects to pass this control parameter to the subroutines it calls, whether you need to use the parameter or not.

Create the Ribbon Interface

I overcomplicated this my first few tries at it. Which is what a lot of people on the internet do. If you want your add-in to have its own ribbon elements, you create those elements using CustomUI Editor (referred to above) within a regular *.xlsm file. When you are ready to finalize it as an Add-In you just save it as an add-in and activate it. The same custom UI that showed up in the original *.xlsm file, will then show up all the time. No fuss, no muss. No Workbook_AddinInstall() code, no SetCustomUI(longComplicatedXMLString), no nothing, it just works! You do have to look at a bit of XML code, mind you.

But first, close down your *.xlsm file and open up CustomUI. Open up your *.xlsm file from within CustomUI. Go to the Insert menu and select Office 2010 Custom Part and, likewise, Office 2007 Custom Part. Now it is time for something ugly. Here is the xml code I used in my project:

For 2010 (customUI14):



For 2007 (customUI):



If you look at the <mso:button> tag you'll notice the launchMyForm bit.  That's the name of the procedure that is to be called.  One parameter is passed, namely, the a reference to the ribbon control that called the subroutine.

Save As an Add-In

At any point, you can turn your *.xlsm project into an Add-In.  (You can even continue editing it while it is an Add-In.)  Go to File, Save As, and select Excel Add-In from the file type options in the SaveAs dialog box.

Friday, August 3, 2012

AutoDesk Design Review 2013 (0x2: Failed to download component.)

I said yes to updating my Design Review (2011) to the current version (2013) and got the above error message.  The solution in my case was to download the MSI file.  (Note:  click on the language you want under the Choose Your MSI column after instruction 2 at the linked location.)

After unzipping the .zip file you can just run the Setup.exe file.  The principle difference is that the downloading has already taken place and the setup program will look at your local or network location for the relevant files to install.

Now, to use Design Review 2013 without AutoCAD installed, you need to install TrueView.  If you have a version of AutoCAD installed on your computer which is temporarily without a current license (but it is installed on your computer), you'll need to explicitly tell Design Review to use TrueView.

To do so:

Within Design Review 2013, go to the main drop down menu (big "D" in top-left)
Options...(at the bottom middle right of drop down)
→ Click on the DWG Viewing tab
→ and check the box beside Open DWG files using DWG TrueView
→ Click OK.

Friday, July 6, 2012

Unfolded Approximation to a Hemisphere

The animated GIF featured below are a sort of slide show which give increasing levels of precision in polyhedral approximations to a hemisphere. The SVG one works properly in Google Chrome, but it may not work in all browsers. In theory, you could print these out, cut along the outline, and fold along the internal lines and you would have a polyhedral approximation to a sphere—if you use a lot of tape and your very best dexterity. More precisely, all of the corners would be coincident with the hemisphere being approximated. Notice that as the number of fold lines (latitudinal-ish lines) increases, for a given number of segments (longitudinal-ish divisions) the extent of the unfolded approximation approaches the grey line, which (probably) represents the limiting extent of these approximations. Specifically, this circle has radius of πr/2 or one fourth the circumference of the approximated sphere.
I wrote the code which generates this sequence of pictures in Maxima, a computer algebra system. The code is below and you can modify the range variable near the end to change the level of precision in the resulting pictures. The pattern for each range item is

[number of segments, [starting latitude divisions, ending latitude divisions]]

The "segments" are longitudinal segments like an orange. The latitude divisions are the number of latitude divisions in each of the segments.

Thursday, June 28, 2012

Mathematics: Principles and Formulae

I gather that when most people think about mathematics, they think about two things, and in this order: numbers and formulas. In doing mathematics, most students are quick to add something else, though not by name: recipes – usually grossly oversimplified recipes. It is this tendency that runs them into so much trouble and many are the able teachers of mathematics that have tried to slap them (figuratively) into using something better, namely, principles.

A famous work of Sir Isaac Newton was titled, in Latin, Philosophiæ Naturalis Principia Mathematica ("Mathematical Principles of Natural Philosophy"). But you don't need to be as smart as Sir Isaac Newton to understand the importance of principles as superior to recipes. First of all, what's the difference?

A recipe is a sequence of steps or components with no (particular) reference to reasons. Computers do recipes - algorithms. Students that try to be computers (and nothing more) short-change themselves because there is already a much more efficient tool for that work than themselves. (And thus it is little wonder they complain about the pointlessness of the learning they are engaged in. Often of their own accord, they are not learning the really important lessons from mathematics that they should be.) Principles help me to justify, or at least partly understand, the steps of the recipe and apply it usefully. The ability Principles are one of the key advantages you have over your calculator.

People try to simplify the setting up and solving of proportion problems. They do it this way: "cross-multiply and divide." They don't really know why. It is only a recipe to them. How do you know which numbers to multiply and divide? It increases the number of things to memorize and (I have observed) it distracts people from the real principles. Forget about cross-multiplying and dividing. If I know a/b = c/d, I do what I always do with equations: do "opposites" and do the same thing to both sides (subject to the constraint that the operation must be permissible on the values in the equation). If I want to isolate a, I have a divided by b, so I do the opposite of divide by b, namely, I multiply by b, and I do it to both sides. The end. "Cross-multiple and divide" doesn't bridge well into solving other forms of equations, is unnecessary, gives little clue as to why it is supposed to work, and increases the number of things to memorize. How do you solve an equation?
  1. Do "opposites."
  2. Do the same thing to both sides.
  3. Eliminate any apparent solutions which do not satisfy the original equations or the constraints of the problem (which may be implicit, such as, "negative area is not allowed").
Throw "cross-multiply and divide" and its ilk into the garbage bin.

Here are a few very generic principles to apply to problem solving:
  1. Don't ask "What should I do?", ask "What is true?"  You won't make very much ground figuring out what to do if you haven't established a few things that you know about the situation. If you want something to do, then do this: find out more true things and organize them. This often starts with writing down some given information. Perhaps you print a few formulas and consider which variables you know and which you don't. Ask yourself, "Do I know any other equations that involve these unknowns? Are they valid in this situation?" Don't worry too much about whether an equation is going to give you the final answer, but concern yourself with whether it actually holds true in the situation. Thomas Edison didn't invent so many things by only trying things he knew would work. You also will write down formulas and find that they do not help you solve the problem – even though they are true and valid in the situation, they might not be useful to you. It isn't a mistake (yet), to write it down as a tool in the toolbox.
  2. Don't ask "Why can't I do this?", ask, "Can this method or statement be proven correct?"  People want to apply rules that are applicable to one type of operation to a completely different operation. They see that they are allowed to make the statement a (b + c) = a b + a c and want to know why they can't say log(b + c) = log(b) + log(c).  This is one or both of two fundamental misunderstandings. Either it is assuming that the parentheses have the same meaning in both cases (which means they don't know what a function is and certainly not what a log is) or they misunderstand the importance of proof. This later misunderstanding in particularly important. Often the student assumes that a similar appearance means a similar treatment is permitted. The fact is, the statement a (b + c) = a b + a c, where a, b, and c are numbers can be proven. Our acceptance of the statement is based on proof from simpler principles. We don't "just know" that that's true (although this example is among the more intuitive that I could have chosen). Similarly, we don't "just know" that the statement  log(b + c) = log(b) + log(c) is true. If we can prove somehow that it is true, then we know we're allowed to make this expansion. On the other hand, if we can find a counter-example, we know it is not true. So here's a counter-example to the above (very, very silly) suggestion: log(10 + 10) = log 20 < 2, but log(10) + log(10) = 1 + 1 = 2; so they are clearly not equal. 
    1. This doesn't mean you need to do proofs, but you should be satisfied that a proof exists for what you are doing and that you are not assuming something which cannot be established.
      1. (Did you notice yourself proving your own work while you applied that principle? Tricky, huh?)
  3. Look for patterns. Patterns have some kind of rule behind them. If you can determine the rule that makes the pattern, it may reduce work and memorization required. The rule will be more generic than the sampling that you noticed the pattern in – otherwise, it is a false pattern, because a counter-example exists. Don't assume the pattern is real. Test it, prove it, disprove it, as may be necessary.
  4. Make analogies. The main reason for teaching the principle of proportional triangles is not to teach students how to solve triangle problems – although there's good uses for that. Proportions are so common place that everyone should get comfortable with the archetypical example: proportional triangles. There are analogies between force and momentum, between torque and force, momentum and rotational inertia, fluid pressure and electricity. Some things are more intuitive to us than others and if an analogy exists between something you are familiar with and something you are not, it can help you with a "working understanding" of the unfamiliar concept.
Understanding principles helps you to evaluate proposed solutions or solution methods. Recipes can't do that very effectively. Recipes also don't work well on a "new to you" problem. Knowing principles is a key point of difference between people who know how to implement a given method of solution and someone who can actually develop a method of solution to a problem they have not previously seen. This is true problem solving. Perhaps the most important thing anyone can do to improve their problem solving ability is to prize principles like gold and recipes as mere silver, maybe only bronze.