streaming cores

Qt DevDays 2010 (#qtdd2010)

This year I've attended Qt DevDays again. And I've found a much bigger venue than previous years. The event is still growing. Roughly 1000 attendees this year. According to the organizers figures double every two years. DevDays moved outside of Munich half way to the airport which is not too bad because you don't have to spent an hour traveling into the city centre anymore. So I was on time to listen to the keynotes and had a chance to grab a coffee before.

Two main topics dominated the event: Qt Quick (Qt User Interface Creation Kit) and the planned Open Governance model for Qt. The first technical topic was demonstrated by Lars Knoll in his keynote and presented in several sessions both days. Basically it's NOKIAs technology to address the app development for all its mobile platforms. The web page reads:

Qt Quick (Qt User Interface Creation Kit) is a high-level UI technology that allows developers and UI designers to work together to create animated, touch-enabled UIs and lightweight applications.

This is an interesting strategical change towards lightweight applications a.k.a. apps to address mobile (NOKIA) devices. Because C++ is to heavy for that, they've added QML "an easy to use, declarative language" deeply integrated with the Qt Creator IDE of course. Obviously, the Qt technology stack will be driven by trends in the mobile world. Feature development for niche *nix platforms will phase out.

The Open Governance model was discussed in the keynote by NOKIA CTO Rich Green and in the Qt Labs. It's good to see a company like NOKIA embracing open source and their community. It's not only discussion that will move into the public, also the QA process will be opened. In the Labs, I got the impression that many details need to be fleshed out. It's not implementation ready but you can clearly identify the trend. And I don't believe it will be easy because this means true change.

Speaking of QA reminds me on something: On day three I've listened to Rohan McGovern in his talk about the "The Qt Continuous Integration System". It was a really impressive and very interesting presentation. Thumbs up, Rohan! It confirmed my experience with QA and CI: choose your tools wisely, dedicate time to it and prepare for a lot of work!

Programming massively parallel systems (FPGA vs. GPU)

Modern Graphic Processing Units (GPU) experience a big hype about their stream processing capabilities. In fact, GPU companies like AMD (ATI) and NVIDIA invest serious money to transform their former rasterizer hardware into general purpose computation engines. General Purpose GPU (GPGPU) is en vogue and attracts individuals and businesses from completely different application areas. The term GPGPU is synonymous for number crunching, high-performance/super computing, financial monte carlo simulation, etc.

How comes that popularity? The top three reasons for the fast adoption of GPGPU are:

Mainstream availability of the hardware.
While FPGAs are unknown to many people, although they're powering many products. Almost everybody who has used a computer knows about the existence of a graphics adaptor and where to get them.

Low costs (COTS).
GPUs are cheap. Their market is large and very price-sensitive. This is probably not true for top-notch products but even high-end GPUs are low priced compared to super-computing hardware or big FPGA devices.

Well known programming model.
NVIDIA CUDA, ATI Stream and Chronos OpenCL are C-style languages. Their model is similiar to tradional imperative/sequential programming languages. It fits the mindset of many programmers and benefits from a large developer community. The FPGA development flow is fundamentally different, requires hardware knowledge and has a steep learning curve.

Graphic Processing Units (GPUs)

The evolution of GPUs is rooted in the 3D graphics pipeline and was heavily disrupted by the introduction of shaders. Shaders are small programs written in a low-level assembler like language. As the name implies, shaders where used to process pixels and vertices before being rasterized in graphics memory. Figure 1 sketches the general dataflow. Big/small arrows indicate bandwidth capacities between pipeline stages. Vertex and pixel processing stages are composed of several sub-steps. I've omitted them for brevity.

Note, that the GPU can leverage its peak performance only if the data throughput is routed from/to its tightly connected video memory. The bottleneck is obviously the interface to system memory. But this constitutes no penalty for a graphics system. The asymmetric throughput is optimized for generating high resolution images with frame rates greater 30 fps from 3D-primitives. (To define a rectangle of arbitrary size you need two coordinate pairs (x1, y1) and (x2, y2) but the amount of pixels covered by the rendered rectangle on screen (in display memory) is proportional to width times height. You're getting the idea.) Textures and vertices are uploaded in advance before being fed into the pipeline. Each pipeline stage increases the amount of data by adding additional information like surface normals, texture coordinates etc.

Since the fixed-function pipeline was limited to a certain set of pre-defined operations a more flexible and programmable pipeline was derived by introducing vertex and pixel shaders (see figure 2). Shaders were a big win for GPU developers. Programmers regained control over the graphics pipeline and GPU vendors were forced to disclose information about the underlying hardware yielding better and more efficient software.

Pixel shaders are often called kernels because they apply the same set of operations to every pixel without knowledge of neighboring pixels. (Vertex shaders work in a similar way but process vertices instead.) This data independency makes them very powerful because multiple kernels can run at the same time without interference. Thus shaders implicitly scale very well (in terms of parallelization) by adding more hardware processing resources. And that's exactly what GPU manufacturers did to meet the demands of their customers: they added more shader processors, introduced vector operations and supported standard floating point formats. They effectively created a new platform for data parallel stream processing.

Field Programmable Gate Arrays (FPGAs)

FPGAs never had a fixed processing pipeline. These devices were not designed for signal or image processing in the first place. Early FPGAs served as glue logic or replaced standard logic ICs. Hence they are categorized as programmable logic.

The internal layout of FPGAs is dominated by a large routing matrix capable of connecting thousands of tiny processing elements called lookup tables (LUT) or function generators (FG). They are arranged in a rectangular grid sometimes called fabric (see figure 3). Each LUT or FG has one or more dedicated flipflops (FF) for storing data or creating deeply pipelined designs. A LUT can generate every logic function from n-inputs (where n is vendor dependent and ranges from 3-6). It can also have accompanying multiplexers or fast lines for carry signal propagation to support arithmetic operations. The routing matrix and LUT functions are freely programmable enabling the FPGA developer to build complex logic. Additional on-chip memory (of several MBit size) can be used for FIFOs or buffering purposes. Lately Multiply-Accumulate (MAC) units were added to the fabric to gain market share of traditional DSPs. High-end devices contain approx. 1000 MAC units. Running all of them in parallel leads to very impressive performance specs.

In theory, these building blocks enable you to build entire CPUs or even GPUs as long as there are enough LUTs. But the evolution of FPGA's went a different way. Programmability comes at high costs. Imagine all the transistors for configuration of the routing matrix or LUTs. And who will start to implement a CPU if you can get microcontrollers with full development toolchain support for single-digit bucks? FPGA vendors reacted by placing commonly used blocks like memory controllers, CPU cores, Ethernet MAC and high-speed serial transceivers (e.g. for PCIe) close to the fabric. No more bandwidth bottlenecks or memory shortage. This clever mix of high-speed IO and arithmetic/logic units turned FPGAs into todays reconfigurable System-on-a-Chips (see figure 4).

Many ventures tried but two companies dominate the market today: Xilinx and Altera. Compared to GPU vendors, these companies operate in a completely different market segment and their target audience is not the end customer. They compete with ASICs, ASSPs and digital signal processors (DSP). You will find their FPGAs in many electronic systems, primarily in embedded systems e.g. backbone telecom routers. FPGAs are complex semiconductors tailored for specific applications. However, these devices remain freely programmable and reconfigurable. It's the designer defining the internal operation and that makes them so valuable and unique.

Summary

Both technologies, GPU and FPGA, are built for stream processing. Depending on the problem you're facing one or the other may solve it better or worse. Development cycles are usually shorter for GPUs because of their well-known programming model. FPGAs perform better in real-time environments when latencies need to be low. The most significant aspect is that a compiled FPGA design results in real hardware.

But it's not only technical factors constraining your design decision. It requires developers with very specific skills and experience. And those are hard to get...

Altium NanoBoard 3000 - In The Box Experience

This series of posts is dedicated to the Altium NanoBoard 3000. I had the opportunity to play with it and like to share my experience with you.

The NanoBoard was released in September 2009 by Altium Ltd. It's a rapid prototyping platform for digital electronic designs consisting of an evaluation board, design software and royality free IP for use in the onboard FPGA. In short, all the tools you need to start implementing your ideas. You can stop reading now, buy it from e.g. Newark for $395 and get started. The 3000 is part of Altium's NanoBoard family a complementary product line besides the well know EDA tools. Maybe you're already working with Designer.

When I received the delivery and removed the outer packing I was impressed. Note, I had not opened the box yet. It reminded me of some consumer lifestyle product. It could have been a mobile phone or high-end notebook. Apple is well know for such a kind of great packaging. Not bad for an evaluation board!

Opening the box reveals the board + software. The black PCB with gold contacts has an undeniable elegance. Underneath is a separate box containing accessories, desktop stand, speaker board, IR remote (yes, a remote control) and the power supply.

The Quickstart Guide provides instructions for mounting the desktop stand and connecting the speaker board. After 10 minutes I had the final setup sitting on my desk. The software is shipped on a DVD. Installation on Windows completed successfully after a couple of minutes. No license is required. The eval board is your license/dongle.

Without having worked with it, this is clearly a highlight among the evaluation kits on my shelf. It's obvious that this product was designed by a team of professionals with a precise and common vision in mind. Every detail has been fine-tuned, from the packaging to the PCB. A good example for holistic product design. Thumbs up.

And now, you know why it is worth to consider the in the box experience. More details in the next post...

Embedded Events and Exhibitions 2010

In preparation for coming week's embedded world I've compiled a list of all events related to embedded computing and/or electronics for this year.

I'm sure it's not complete. So feel free to leave a comment and I'll extend the list.

embedded world 2010
Nuremberg, Germany
March 2 - 4, 2010

electronic displays 2010
Nuremberg, Germany
March 3 - 4, 2010

IIC China Spring 2010
Shenzhen, China
March 4 - 5, 2010
Chengdu, China
March 11 - 12, 2010
Shanghai, China
March 15 - 16, 2010

DATE 10
Dresden, Germany
March 8 - 12, 2010

Intel Developer Forum 2010
Beijing
April 13 - 14, 2010

ESC Silicon Valley
San Jose, California, USA
April 26 - 29, 2010

The Embedded Masterclass
Cambridge, UK
May 6, 2010
Reading, UK
May 11, 2010

ESEC, 13th Embedded Systems Expo
Tokyo, Japan
May 12 - 14, 2010

ERTS² 2010
Toulouse, France
May 19 - 21, 2010

TechEd North America
New Orleans, Louisiana, USA
June 7 - 11, 2010

ESC India
Bangalore, India
July 21 - 23, 2010

IIC China Fall 2010
Wuhan
September 13 - 14, 2010
Dongguan
September 16 - 17, 2010
Xi'an
September 20 - 21, 2010

EIE-2010
Moscow, Russia
October 26 - 28, 2010

TechEd Europe
Berlin, Germany
November 1 - 5, 2010

electronica
Munich, Germany
November 9 - 12, 2010

X86 Single board computers gaining momentum

With the growing popularity of single board computers (SBC), especially those based on x86-compatible CPUs, more and more applications pave their way into the embedded systems domain. What we're experiencing is the reach of traditional desktop-like applications into areas, that were restricted to full custom developed and RTOS-driven hardware a few months ago.

SBC with enclosure

Vice versa, there's a complementary trend of traditional embedded applications moving from custom hardware and niche software platforms to single board computers powered by main stream operating systems e.g. Linux or Windows embedded. Driven by narrow market windows, cost pressure and short technology cycles, product makers try to keep up with the pace by using pre-designed and verified hardware platforms and/or Open Source software frameworks hiding the underlying complexity from their development teams.

This convergence from both sides of the embedded spectrum has gained momentum with the introduction of the Intel Atom architecture (and its compatibles) early last year. Its low TDP in combination with well-known interface standards e.g. USB, Gigabit Ethernet or SATA has lead to fanless and compact board designs, perfectly suited for single board computers used in industrial applications.

So called computer on modules remove the burden of designing an entire processor platform from hardware engineers and shift the focus back to their core business: implementing solutions for your customers. A carrier or base board with standardized connector, e.g. via ETX/COM Express, to the processor module provides application specific functionality through interfaces like USB or PCIexpress. The same applies to the software development approach: a commodity Linux distribution replaces board support packages, embedded network stacks and error-prone third-party libraries. It should be obvious, that x86-compatibility gives you access to a giant amount of existing software and one of the largest developer communities today.

Computer On Module

Let's take an image processing system (could be used for quality inspection of an assembly line) as an example. The task is to capture N-images every second, perform a feature detection in the image, store the result and assert a signal in case of missing features to sort out items not conforming to spec (see following picture).

The traditional embedded approach would be to connect a camera module e.g. via Camera Link to your custom-designed PCB equipped with a decent digital signal processor (DSP) and an output interface to connect to the real world e.g. serial/parallel GPIO. While there is nothing wrong with that, the question is: Is it worth designing (and maintaining!) full custom hard/software? Do we need hard realtime? How fast can you deliver the product? And at which costs? Basing our system on a single board computer instead, leads to the following simplified system level architecture.

We use a CCD based camera module that supports continuous streaming over USB like the Lumenera Lm075. Isochronous transfers ensure a bounded transmission latency. Drivers and SDK are available so that image processing algorithms can be developed and tested right from the beginning on every PC with USB connectivity. Yes, this is a huge benefit! If you've ever had to setup a vendor specific DSP development environment, you know what I'm talking about..."Hell! Why can't I connect to the evaluation board!"

The good news is, that current x86 architectures implement powerful DSP-operations like Streaming SIMD Extensions (http://en.wikipedia.org/wiki/SSE2) and are capable of dealing with double precision numbers which is still not given for most DSPs. Examples how to leverage these instructions for algorithm acceleration can be found here (Using SSE for image processing).

Once the core algorithm is implemented, the outcome of each processed image needs to be (1) logged and (2) in case of errors signaled to external hardware dealing with the failing item e.g. sorting it out. The first task can be accomplished by simple logging to a file or, thanks to the Ethernet port, by storage in a SQL database. External signaling is slightly harder but can be done via SuperIO chipsets or GPIO pins if available. Usually, this involves another level conversion or buffering e.g. to 24V for industrial automation systems. And we're done...

Well, what I've left out for the sake of brevity is the timing part. The critical path from image acquisition to external signaling needs to be carefully evaluated and finetuned to meet your timing requirements. Hard real-time is only available with RT-patched kernels but can be accomplished as well.

The computing architecture described in this post is not restricted to image processing applications only, in fact it can be applied to many problems in the embedded systems domain. It integrates perfectly into corporate IT networks due to its Ethernet capability and can be administrated/monitored remotely. The benefit of using an architecture similar to the given example is, that the overall system design effort shifts noticeably to the software side, where your expertise is located, in the implementation of core business logic based on a proven and standardized platform.

Recruiting the right people

Recruiting people is always fun. Really. Not because I'm a fan of the puzzles that you need to solve during the interview process. That's quite typical for big companies like Google. No, the reason is pretty simply: I like to talk to people. From time to time I need to help out HR folks at a career fair and so it happens that I talk to graduates or young professionals seeking for a job in the tech sector. The atmosphere there is more relaxed than in the sitting-in-the-office-and-being-observed interview situation. People show up at your booth, you start with some small-talk and suddenly you're in a conversation. (Or not, depending on your counterpart.)

Neither a resumee nor a telephone interview can reach the level of face-to-face communication. You can look into the person's eyes, find out how they think about technologies, what's important for them and what they expect from their future employer. Wait a minute, this is an interview?! Right. You haven't noticed? Good. Of course, this is just an early stage of the whole procedure but it's a very effective way to reject 50% of the resumees that get sent to you where the applicants proved to be unable to read and understand your job description.

OK, you're dedicating an entire day for talking to applicants. But how can you know if it's worth enough to schedule a follow-up interview? I've developed a gut feeling over the years and some techniques:

You're looking for self-motivated people. It's when you ask for the last project and they get excited. They're saying: I love what I do and I'm 100% convinced. See also intrinsic motivation.

You need curious people. Do they ask the right questions? If they do not have a serious interest in your products it's a waste of time for both sides.

You want quality. I don't know about you, but my threshold is very high. I found it very useful to ask for samples from their last projects. This gives me the chance to discuss important parts of it in detail in order to get a better understanding of their problem solving skills.

Yes, it's June 2009, the world is in the middle of a recession, the news are full with layoffs, entire industries are collapsing and I'm writing about hiring. But each downturn also has its winners. This is a chance for small and medium sized companies with long-term strategies and innovative products on their roadmaps. It's the chance to attract the best and creative people by demonstrating your strength: You are still hiring. And it feels good.

In times of economic growth and rising stock markets when finding good engineers is unlike harder, you may end up in the following situation: the candidate is fresh from college, seems to fit the job and you're getting into the "salary topics". Now things are getting a bit odd. It turns out that there is another offer from big-company and this and that bonus etc. everything is urgent, headhunter, younameit.
Emergency stop!
You can play the money game but is it worth playing it? Remember technique #1? It's not! I'm not saying "Don't pay decent salaries." The opposite is true. You need to pay decent salaries in order to get the best people. It's even worse. You also need to offer top-quality working conditions (but that's another story)! I'm saying you're not the two year projects driven consulting gig hoster throwing ridiculous amounts of money at applicants just because of urgent resource requirements.
Let him go! This can be very disappointing but you should be thankful because you found it out so early.

embedded world 2009 - personal retrospective

You can guess it from the title. I've attended embedded world this year (again). This is my personal review from Europes biggest exhibition about embedded technologies held in Nuremberg (Germany) from March 3rd to 5th.

I like the narrow focus on embedded hardware, software and services. The whole event fits in only four pavilions but has plenty to offer. Unlike some big events (e.g. electronica, Cebit) it's still growing: a 25% increase in exhibitors compared to last year. (I've found no official statistics about the visitor numbers, yet.)

As last year I took the train and arrived around 11am in Nuremberg. Passing by the first busy booths gave me a good feeling. It seemed like recession was dispelled from these pavilions. Maybe I'm wrong, but the general mood of the visitors didn't show a sign of crisis. Of course, everybody is aware of the exceptional situation and I expect more people getting laid off in the electronics industry but despite of all bad news the booths showed high load.

I've noticed an interesting trend going on in the computer on modules (COM) business. Multiple vendors are jumping on the bandwagon of increased system-integration. In order to reduce the number of components, save PCB space and power they start to offer modules with on-board FPGAs. The FPGA is connected through a 1x PCIe lane to the chipset and provides external (serial) connectivity for I2C, CAN, Ethernet etc. This sounds contradictory to reducing power? Wait. The best news is that you have access to the unused FPGA fabric and can insert your logic there. Even better, get rid of the CAN core and friends and occupy it all!

In the software pavilion I've went to the Trolls (aka Qt Software) and talked about the latest Qt release. Starting with version 4.5 the Qt framework is also offered as LGPLed package. That means you can link your closed source applications to the library without paying any license fees. That's good news for independent developers and micro ISVs. I was wondering how Qt Software is making money now. I got a lengthy explanation which can be summarized to: More developers, more mobile applications, higher Nokia phone sales. Nokia pays the bills. Additionally, the Qt Extended stand-alone product is discontinued. The last release will be 4.4.3. All Qt Extended features will be moved later into the Qt framework.

So far. In case things should get worse next year we can still ask Gov. Schwarzenegger for a keynote...erm click here.

New FPGAs for high-volume applications

Recently Xilinx caught my attention with the announcement of their next generation Virtex and Spartan FPGA platforms. Having worked with the Spartan series I was really excited to see the version number double (!) from 3 to 6. Yes, say hello to Spartan-6. And...Virtex-6 of course but I'm focusing on the low-cost family for now since it got the overall (and long awaited) face lift.

The first three things I've noticed were (1) the addition of serial transceivers, (2) an increased MUL : LUT ratio and (3) memory controller/PCIe endpoint blocks. Finally! Finally, Xilinx made it and added built-in support for high-speed serial communication. No need to look over with envy at Altera or Lattice with their low-cost families anymore...wait a minute. Altera announced new Arria II GX devices and Lattice is about to release the ECP3M family. But don't expect a comparison now. More or less, the three points apply to all low-cost families across vendors:

(1) The Spartan-6 platform is split into two classes: seven devices without high-speed serial I/O and four devices with 2 / 4 / 8 GTP low-power transceivers/PCIe. While Altera and Lattice are pushing the second generation of low-cost devices with serial transceivers out of the door it's Xilinx first device in the price-sensitive market. And it's time. High-speed serial connectivity is becoming the emerging standard in complex embedded systems. And believe me systems will get even more complex and demand more bandwidth in the future. To overcome complexity and reduce development time vendors need to support a well established communication standard. And what USB is for the desktop market will PCIe become for embedded/industrial applications.

(2) Marketing folks identified mass market audio and video applications as typical innovation areas. These applications are driven by digital signal processing (DSP) and require lots of computing power. So give 'em multipliers + accumulators aka DSP slices aka (sys)DSP blocks! That's my second point: an increased number of multipliers per logic resources. Up to 182 DSP48E1 slices in the biggest Spartan-6 devices are waiting for your DSP algorithms. And DSP is everywhere especially inside todays FPGAs. Or are you (mis)using them for glue-logic?

(3) Memory controller blocks are just another step towards systems-on-chip. They'll become handy when synthesizing a soft CPU core into the fabric or when large amounts of data need to be buffered. Lattice had them already built in the ECP2M Programmable I/O. They really set the benchmark in the price-sensitive market then. I can imagine that Xilinx lost some customers to Lattice here...

However, competition goes on. You can meet all three FPGA vendors at embedded world in Nuremberg this week from Tuesday to Thursday (3.-5 March 2009) and give them feedback on their latest technology.

About templates and optimization

Aside from reverse engineering FPGA bitstreams I've finished (Yes!) another pending programming project last week. I was helping out a fellow programmer, let's call him Bob, who was busy doing other things and had no time to hack away a set of high-prioritized items from his TODO list. So I pulled these items from his list over to mine. I have plenty of experience doing these troubleshooting jobs and I always feel like a surgeon carefully inspecting the innards (source code) of the patient (application) before cutting out tumors (bugs) or implanting organs (patches)...

Ok, to come back to the problem: I was supposed to extend a desktop application that was designed and written from ground up by Bob two years ago in C++ (using Qt). Hmmm...C++ and templates. I can hear your brain working now. Right, we're not talking about document, XML or whatever templates here. It's the well known C++ programming language feature (see Template Metaprogramming). I use it regularly. It's a great feature if the compiler fully supports it. If you're familiar with templates feel free to skip the next paragraph.

Templates implement the concept of parameterized types in C++ (Bruce Eckel, Thinking In C++). It's a syntax extension that tells the compiler how to define a type from a generic type declaration. Wow...that was abstract. How does it work? The first time you instantiate a template with a given type (as parameter) in your code this parameter is inserted by the compiler into your template declaration and creates an entirely new type. Ideally templates are designed in such a way that you can throw almost every type on them (as parameter). Yes, even user-defined types. Think of it as another way of reusing code besides the OOP based inheritance approach. The syntax can also be applied to function definitions (aka function templates). Templates are perfectly suited for container classes like lists, queues etc. of arbitrary objects. Personally, I've used templates in digital signal processing algorithms to create number format independent filters amongst others. Other classic/good examples are the C++ Standard Template Library (STL), the Boost C++ libraries or Intel's Threading Building Blocks (TBB).

The challenge with templates is that it's very convenient and easy to use them but significantly harder to create them in a structured and maintainable way. Did I mention that templates are entirely implemented in header files? That may become an issue as I'll explain later. Now guess what happened...the app I was working on was full of templates. Driven by the idea of increasing runtime performance Bob applied the template concept to nearly 100% of the data path related code. His intention was to let the compiler optimize e.g. (inline) nested function calls introduced by the object hierarchy and flatten them out at compile time. No virtual function lookups and stuff in the binary...anymore. Highly instruction level optimized code. The perfect solution for maximum performance...

Problem1: The template concept is no silver bullet for your performance problems. The 80/20 rule still applies: 80 percent of the execution time is spent in 20 percent of your code. (Or was it 90/10?) Use a profiler and analyze your code carefully before delving into advanced and unreadable templates of templates of template constructs.

Problem2: While it might be a (good) academic exercise to create an entirely template based object hierarchy to gain a deeper understanding of the language concept it's probably a bad idea to apply this pattern to your entire framework. First, this makes it hard or even impossible to put your framework later in a library and second, the guy next door (patching your stuff) appreciates readable code.

Problem3: Have you ever tried to debug and/or patch this stuff? When 98% of the files are header files you'll trigger a recompile of the entire project just because you've changed a variable name from i to tableIdx, and not to mention the weird error messages you might get in seemingly unrelated sections of the code (although compilers got better and precompiled headers may reduce the pain a bit). Yes, I'm exaggerating here but you're getting the point.

Don't get me wrong. This is no rant against templates! Letting the compiler write type-safe code for you is of great value. Also, generic datatypes tremendously reduce the amount of duplicated code. But be careful when using this powerful language feature. Use it only where applicable i.e. to maximize code reuse, restrict it to subsets of your framework, keep in mind who might read/use the code and please please don't misuse it for your global optimization strategies.

You want to know the end of the story? Well, just another surgical intervention.

Stepping into parallel computing

While writing my last post I remembered how I came across parallel programming the first time. I was at university and at that time looking into cracking passwords from hashes e.g. on UNIX-like systems from /etc/shadow using John the ripper. BTW when saying cracking I mean the real brute-force approach, no dictionaries and stuff. The SUN workstations were too slow at that time. So I bought a bunch of 3G base station processor boards at ebay (see picture) armed with 4 DSPs and a PowerPC. Pretty hot stoff that time (and ridicously cheap, about 10 EUR each). They were used by a huge german company (starting with S) in a development project and sold at the end.

I had no clue about FPGAs that time and DSPs were my natural choice. I was young, I knew how to program in C, I was using an open source password cracking tool and the DSPs simply matched my number crunching requirements. What else do I need more? Erm...board manuals, schematics, professional development tools (yep, no gcc port for TMS320C6X available), debugging cables, probably a VME backplane and more knowledge about these architectures as I found out. Finally I managed to power the board using an old PC power supply and connected to the console port. Woohoo...it was still working and the bootloader was looking for a FTP server to pull the OS image from. My first steps into the field of parallel computing...

To bring the story to an end. I've never run any code on the DSPs due to the lack of board manuals, schematics, professional development tools...you name it. However, I've learned that (1) certain computing tasks can be broken down to a restricted set of arithmetic operations but need to be run at the maximum achievable speed and (2) it makes sense for a group of problems to split work among multiple processors (or cores) to finish computations in less time.

From todays perspective I've discovered another interesting aspect which is more related to the overall system concept. If you take a closer look at the architecture of the telco blade, you'll find similarities to modern processor architectures. There are specialized processing units (4 DSPs) grouped around a central processing unit (PowerPC) on the PCB. Does this remind you of something? No? Replace PowerPC with PPE, DSPs with SPEs and PCB with die and you'll get pretty close to the CellBE architecture (see figure below, taken from "Introduction to the Cell Broadband Engine").

Lessons learned from application specific processor boards lead to main stream processor architectures on a single die. I'm wondering if the telecommunications industry is still the number one driver of the processor evolution. Or is it the gaming industry with its demand for high bandwidth graphics hardware that is most influential on modern processor designs?

Great multi-core resources from Intel

Today I want to point you to the parallel programming resources from Intel. Wether you're new to multi-core development or you're looking for a specific answer to a concurrent programming question, this website is the place to start:

http://software.intel.com/en-us/multi-core/

Even experienced programmers will find it useful to browse through the whitepapers section or take a closer look at the technology demos (source code included). I know that all content is focused on Intel's technology but I like the way Intel cares about their developer community. Erm...I must admit that I'm a little biased towards their technology. Speaking of community, there are also several blogs from Intel employees covering all sorts of programming topics from threading to the latest Intel tools or libraries.

Specifically, I want to highlight the great weekly podcast hosted by Aaron Teersteg. He's the community manager for Threading for Multi-Core. In his podcasts (approx. 15 minutes in length) Aaron interviews his guests about their relation to parallel programming. I'm listening to it reguarly. It's short (but long enough for the daily ride to the office) and gives you a good start into related topics e.g. OpenCL or functional programming.

A last tip for today: Check out the Smoke Game technology demo!

Software backed watchdog timers (WDT part 3)

This is the last post in my series (see part 1 and part 2) about watchdog timers (WDTs). After that one I will stop bugging you with the topic. But probably you've already got the impression I have a certain weakness for WDTs. Anyway let's take this to the next level and find out how to improve the solution from my last post even further.

Right now we have our application directly pinging the watchdog hardware i.e. there's a watchdog module/component linked into the main executable doing all the things required to prevent a timeout. That was easy to do because it only required us to insert two function calls for (de)initialization and one for pinging inside our mainloop. Unfortunately it also introduced a tight coupling between the main application and the watchdog handling logic (see figure 1).

Other drawbacks of this approach are:

loss of flexibility and reuse (OK, reuse is possible but only on the code-level)
adding support for new WDT hardware requires rebuilding the application or may have other side effects
it's harder to verify/test the watchdog handling logic

To overcome these drawbacks we put the watchdog handling logic into a separate executable and introduce a small and light-weight layer that wraps the interprocess communication (IPC) which is now required. I'll leave the choice of an appropriate IPC API to you. It should be non-blocking and easy to use (SOAP is probably a bad idea). Certainly we've added slightly more complexity to the big picture (see figure 2) but at the same time it removes any dependencies from the main application to the WDT hardware and it can be run even when there is no watchdog present at all.

Having done these architectural changes we're now ready to introduce more logic to improve the WDT behavior. We can even add a user interface to the newly created watchdog application. Please note, that the term watchdog application/process refers now to a piece of software sitting on top of the WDT hardware.

The nonobvious outcome of inserting a watchdog process between the main app and the WDT hardware is that we're getting an additional high-level fallback layer without degrading the original WDT functionality. Both applications + OS remain under WDT control (remember this is a piece of hardware hooked up to the reset line). This gives us more opportunities to take action when the main app stops pinging the watchdog process e.g. we could try to kill and restart the main application process once and ultimately initiate a shutdown sequence which is healthier than the hard reset. And that's not all! Further enhancements may include:

Restore the factory settings of the main application after a reboot in order to undo user changes (works only if the watchdog process starts first).
Log all actions / errors or display a warning message on a local display. I recommend to use the operating system log.
Send or broadcast a message before shutdown over the network to a monitoring instance if available.
<put your feature here>

I'm pretty sure you'll have more features to add, but always remember: Keep it simple! It's definitely not the right application to bloat with features.

Somewhat intelligent watchdog timers (WDT part 2)

While my last post was focused on the hardware-side of watchdog timers (WDTs) I will now discuss more high-level/software concepts of WDTs. So, if you haven't read part 1. Now is the time! Done? Here we go.

I'm assuming we have a software powered application that requires a full-fledged operating system (OS) and relies on a bunch of peripheral hardware besides a CPU e.g. harddisks, network adaptors etc. Let's call it server since that might be a valid use case. OK. Our application is supposed to run 24/7 somewhere deeply buried at a customer's site and it's extremely costly to send out tech-support staff for on-site fixes, just to discover that e.g. someone played around with our settings or temporarily disabled the air condition for maintenance causing the system to lock-up.

Being smart and having read part 1 we pull out this timer thing connected to the reset line of our server and we're set, right? Wrong. The problem with this approach is its simplicity and tempting ease of implementation. I understand that you want to get things done and push the box out of the door. But this approach brings further implications we havn't dealt with yet. Remember, our software is not running on a microcontroller. A hard reset should only be the last resort since it puts a lot more stress on all components than a safe shutdown. And what if there is really a broken piece of hardware or the air-condition runs amok and starts heating? It will result in an endless reboot-reset cycle causing even more harm.

Let's tackle the "endless reboot-reset cycle" problem first since it can be applied without changes to the hard reset implementation. The idea is quite simple: we extend the WDT by counting the number of timeouts. If a timeout occurs our application is obviously not running the way it was intended to. So we maintain a counter or some flags (ideally this is done in hardware) and additionally log the time in a non-volatile way. Observing more than X timeouts in a timeframe less than Y seconds (insert appropriate values for your application) will power down the system. Now, human intervention is required for a restart.

I've tried to illustrate this in a loose UML-style statechart (see figure), on the left hand-side the application process and on the right hand side the watchdog process. The dashed line denotes concurrency. State names are written in bold + underline. Lines with arrows denote state transitions. Transitions can be conditional (with label) or unconditional without label. Including the WDT reload/ping of the application process was a bit tricky. Note, the arrow which is overlapping the dashed line. I've found no better notation. If somebody can shed light onto that issue, feel free to comment.

In my next post I will cover the "safe shutdown" issue and why it is important to split this functionality from your core software.

Watchdog timers (WDT)

(Delayed) Happy New Year everyone! My New Year's resolution for this blog is to keep posts shorter than the last one. Looking over it again made me feel that it was a bit too heavy.

Now back to the topic. I'm going to blog about watchdog timers (WDT) today because it was one of the main topics that bothered me for this week. Watchdogs are very common in embedded systems to resolve system/software hangs. These non-interactive systems need a mechanism to automatically reboot or recover from failure states without human intervention. So how does it work? The idea is quite simple. As the name says it's a timer which itself finally boils down to a counter. That counter is part of a dedicated simple circuitry running independently from the processor and continuously counting down e.g. at a speed of 1 tick per second. If the counter reaches 0 a signal is asserted and ... something has to happen. Usually you'll find the following two scenarios:

The hard way: the signal is hard-wired to the reset line and boom ... the system restarts.
An interrupt is triggered and a handler can perform error logging / cleanup operations or initiate a safe shutdown. What if the handler fails? Well, meanwhile the counter is reloaded and starts counting down again but this time going for the hard reset.

I'm sure you've already figured out what to do. To prevent timeouts one has to periodically reload the counter to assure your program is still alive. This is also referred to as pinging.

WDTs can be found in nearly every microcontroller today, on server mainboards or dedicated extension cards with programming support.

In my case it happened to be a Super Micro X7SBE motherboard with on-board watch dog. Since the board is equipped with an ICH9R chipset which contains WDT functionality (as part of Intel's TCO logic) I've expected the BIOS to use that one. Later (about a dozen reboots) I figured out that it's the watchdog in the W83627HG I/O chip. Of course, this is not documented in the mainboard manuals and there's also no driver for the device on the drivers disc (except, if you count the winio library). So, if you're planning to use an on-board WDT: Watch out! Don't walk into this trap.

Upgrading from Virtex to Spartan

Granted, I've left out the important numbers in the title. To be more precise: this post is only for those of you maintaining an existing PCB design containing one or more Xilinx Virtex-II devices.
Many times it happens that you're forced to touch the board again because of discontinued parts (no, not the Virtex-II), minor circuit improvements or feature requests from your customers. So why not upgrade the Virtex-II to a package-compatible Spartan-3A at the same time?
No way!? Why should you do that? Valid points but before I'm going to explain why it could make sense in certain situations I want to introduce the prerequisites for the upgrade:

it works only for the FG256 package (Virtex-II devices from XC2V40 to XC2V1000)
the system frequency must not exceed 300 MHz
your VHDL/Verilog design is without (too many) device specific instantiations or physical constraints

If these (heavy) requirements apply to your design this could be your migration path:

Original part	Potential upgrade part	Upgrade part
XC2V40		XC3S200A
XC2V80		XC3S200A
XC2V250	XC3S400A/XC3S700A*	XC3S1400A
XC2V500	XC3S700A*	XC3S1400A
XC2V1000	XC3S1400A*	N/A

With the extension of the Spartan-3A family in August 2008 Xilinx now ships the entire device family in the FT256 package. The FT256 is slightly thinner but FG256 "compatible". Now, it's possible to replace a small to mid-range Virtex-II by using a large Spartan-3A.

In the table above I did the logic, multiplier and DCM resources comparison for you to find the appropriate replacement candidates (see third column). The second column lists all Spartan-3A devices that have the same amount of CLB logic than their Virtex-II "counterpart" but less multiplier/BlockRAM resources. So if your design doesn't use 100% of these non-CLB resources they might be an alternative, too. That's why I've called them potential upgrade parts.

Now back to the question: Why? I know it's a lot of work and not as easy as replacing just the FPGA in the next board revision. It requires adjusting the core voltage, changing the layout, rewriting constraints, updating production specs etc. etc. But it will pay off in the following points:

lower costs per board
lower power consumption (due to reduced core voltage)
less heat dissipation
more logic/CLB resources for future extensions

Which I consider valid reasons for small to large volume products and especially battery powered applications. Please note, this post is not meant to be a HOWTO or guide nor can I guarantee you that it works for all designs. I wanted to share the idea and research work with you. In case you find this interesting or want to exchange experiences I would be glad to hear back from you.