Designing a simple and cheap temperature logger. Part 4: The junk in the trunk.

I somehow managed to solve the problem of the serial flash memory write block size vs. the erase block size. I just had to write a custom FAT12 filesystem..

As seen before, the Temperature Logger will act like an USB memory stick. There will be at least two files. One with the time stamped temperature measurements and one file allowing the user to set the logger’s configuration.

The configuration file must be accessible by any OS and modifiable. It will configure the temperature measurement period, from one every second till one every 24h (86400 seconds), in seconds. It’s also used to synchronize the PIC’s real time clock and calendar to the computer’s current time and date.

So far, it looks like that:

#THIS IS THE CONFIGURATION FILE
Setting the temperature logging period, in seconds
1<period<86400
Don't forget to add a "#" before the number:
#70
#END OF THE CONFIGURATION FILE

Pretty simple. When it’s saved, the PIC will read the number following the “#” and the time and date of the file modification.

The FAT filesystem is divided into five sections, each of them divided into sectors:

  • The Master Boot Record
  • The Boot Record
  • The FAT (doubled)
  • The Directory
  • The Data

When a file is written, its content will be placed in the Data section. The name and other information will be in the Directory. The FAT will then describe where the different data pieces of the file are located on the disk.

I took the standard sector size of 512 bytes (the size of the PIC’s buffer and the double of the EEPROM’s write block).

The MBR, the Boot Sector and the Directory are 512B from the OS point of view and 4KB on the EEPROM. I need five FAT pages to map the capacity of my EEPROM. Here’s a map of my filesystem:

FAT12 MicrochipAll the sectors occupied by the filesystem sections are 4K wide.

(The help.txt file contains the basic explanations on how to start/stop temperature acquisition and how to erase the data.csv file)

I modified the Microchip’s SD-SPI.c file to be able to talk to my memory, instead of a SD card. The main functions are:

BYTE MDD_SDSPI_SectorRead(DWORD sector_addr, BYTE* buffer)
BYTE MDD_SDSPI_SectorWrite(DWORD sector_addr, BYTE* buffer, BYTE allowWriteToZero)

They both take the address of the EEPROM sector to be written to, in LBA format (ie. the sector numbers) and translate them to the real EEPROM address. The buffer is the 512B PIC’s memory zone to write to the EEPROM or to fill with the EEPROM data.

An if statement chooses the address shift. Everything that’s below the DATA.CSV file will be 12-bits shifted ($02 becomes $2000) and the 4KB zone will be erased before any write occurs. If the write or read is in the zone where the temperature measurements are, the shift will be relative to the DATA.CSV address ($2E000), and looks like:

Address = Address - $2E
Address = Address << 9
Address = Address + $2E000

Which means, for example, that if we read or write the third sector of DATA.CSV, the function will be called with $30 and will result in the EEPROM address of $2E400, which is the third sector of the EEPROM after $2E000.

Of course, any write to this zone by the OS has to be forbidden.

For this, the Microchip USB Mass Storage Device driver uses the same functions as above, but in the usb_function_msd.c file. It uses an other buffer, msd_buffer, 512B too.

Every time the computer OS wants to read or write into the EEPROM, the data is cached in this buffer. Two functions are doing that:

BYTE MSDReadHandler(void)

BYTE MSDWriteHandler(void)

In each of these functions, I inserted a test to know what and where the OS wants to read or write.

In the case of a read, I’m not doing a lot for now (the MDD_SDSPI_SectorRead function already handles it).

For the writes, I first test if the OS is writing the data of config.txt file. If yes, I search for the period string and write the data to the memory. If the OS writes the File Dierctory, I read out the new date of the config.txt file but I write the original contents of the File Directory in the EEPROM instead of the new ones.

For any other write from the OS, I just answer the write was OK, but don’t write the actual data to the EEPROM.

This way, I’m sure the OS won’t fill the memory with anything else but the right data. It’s especially true for OSX, who wants to write Spotlight, Trash and info files in any USB stick it finds.

Here’s a simplified diagram of the modified part of SectorWrite function:

  • if msd_buffer contains the config.txt start and end string -> Extract the period, write the buffer to the config.txt file address.
  • if msd_buffer contains the File Directory -> Extract the time and date from config.txt, if it’s newer than the original date. Erase the msd_buffer and fill it with the original contents, minus the date and time fields. Write the buffer to the File Directory address.
  • in any other case, perform a dummy write.

That way, we save a lot of space in the EEPROM. If it would have been divided into 4KB sectors, with only the first 512B usable, we would have ended with a ≈100KB disk ((($FFFFF-$2E000)/$1000)*$200). Now, even if some space is wasted for the filesystem, we have a ≈840KB drive ($FFFFF-$2E000). It means something like 30 000 timestamped temperature measurements.

Some precisions about the FAT:

The FAT is used to keep track of every file on the disk. Each of the FAT entries matches a sector on the disk (that’s why my FAT is 5 sectors long, we need 5*(512/1,5) bytes to map our 1648 sectors). Here’s my FAT12 capture, as it comes after initialization:

FAT - FAT12 - Temperature Logger - FormatedThe first two entries are sector 0 and sector 1. They’re reserved. That’s why the data zone is located starting $2C000 (each sector is $1000 bytes before the DATA.CSV file and the FAT ends at $0C000).

The three next entries are $FFF,$FFF and $FFF which means that the sector contains one complete file (config.txt, help.txt and data.csv). (It’s easy to understand why small files are wasting space: Every file has to occupy at least one sector, even if it’s smaller than the size of the sector)

When a file is spread on more than one sector, its FAT entry will have the next sector address.

Unfortunately, the entries are coded in 12 bits values, and in big-endian format. If we want to write xyz for the sector 4 and XYZ for the sector 5, we have to write:

Yes, could be easier (it actually is, with FAT16 and FAT32, but our memory is too small to use those formats)

Here’s an example with the data.csv file spreading on 15 sectors (512*15 Bytes wide):

FAT12 - 19 sectors

If we decode the 12-bit addresses, it gives:

FAT12 - Decoded

There is one file in sector 2. One in sector 3. The third file starts on sector 4 and goes till sector 18.

I made a function to add entries every time a buffer full of temperature logs is written into the EEPROM. The 12-bit address management looks like that:

if(FatSector & 1) //Odd Sector Number
 {
 FATEntryPtr = FatSector-1;
 FATEntryPtr = FATEntryPtr + (FATEntryPtr>>1); // *1,5
 FatDWordA.w[0] = ( (FatSector+1) & 0xFFF);
 FatDWordA.w[1] = ( (FatSector) & 0xFFF);
 //We have:  0x yz 0X YZ and we want:
 //          00 yz Zx XY
 FatDWordB.v[0] = ((FatDWordA.v[1]&0x0F)<<4) | ((FatDWordA.v[0]&0xF0)>>4);
 FatDWordB.v[1] = ((FatDWordA.v[3]&0x0F)) | ((FatDWordA.v[0]&0x0F)<<4);
 FatDWordB.v[2] = FatDWordA.v[2];
 //The FatDWordB Contains what we want to write to the FAT, for the current //Sector and the next one.
 msd_buffer[FATEntryPtr] = FatDWordB.v[2];
 msd_buffer[FATEntryPtr+1] = FatDWordB.v[1];
 msd_buffer[FATEntryPtr+2] = FatDWordB.v[0];
 msd_buffer[FATEntryPtr+3] = 0xFF; //And write the end of the file for the //next sector entry
 msd_buffer[FATEntryPtr+4] = 0x0F;
 }
else //Even Number
 {
 FATEntryPtr = FatSector;
 FATEntryPtr = FATEntryPtr + (FATEntryPtr>>1); // *1,5
 //We want to write the next sector address in the current one, plus the //end of the file on the next sector
 FatDWordA.w[0] = 0x0FFF;    //End of the file for the next sector
 FatDWordA.w[1] = ( (FatSector+1) & 0xFFF); //In the current sector: the next //sector address.
 //We have:  0x yz 0X YZ and we want:
 //          00 yz Zx XY
 FatDWordB.v[0] = ((FatDWordA.v[1]&0x0F)<<4) | ((FatDWordA.v[0]&0xF0)>>4);
 FatDWordB.v[1] = ((FatDWordA.v[3]&0x0F)) | ((FatDWordA.v[0]&0x0F)<<4);
 FatDWordB.v[2] = FatDWordA.v[2];
 //The FatDWordB Contains what we want to write to the FAT, for the current //Sector and the next one.
 msd_buffer[FATEntryPtr] = FatDWordB.v[2];
 msd_buffer[FATEntryPtr+1] = FatDWordB.v[1];
 msd_buffer[FATEntryPtr+2] = FatDWordB.v[0];
 }

I’m using the msd_buffer to save the FAT during temperature acquisition (and the gDataBuffer to save the temperature logs) between two writes to the EEPROM, when on battery power.

Every time a gDataBuffer is full, it’s written to the EEPROM in the data zone and a new FAT entry is added to the msd_buffer. Once this one is full, it’s written to the EEPROM at the FAT address.

The variable FatSector contains the current sector in the FAT. FatDWordA and FatDWordB are temporary Double-Word variables used to compute the three-bytes-12bits FAT entries.

This is MCC18 PIC18 code, based on Microchip USB and MSD libraries. I wanted to share this snippet, as I couldn’t find any good PIC code example on the internet. I’m sure it could be optimized and simplified, but like that, it’s maybe easier to understand how it works.

I think it’s all for now. My next step will be to consolidate my program and code the power-managed temperature acquisition. I already have the Temp. sensor, EEPROM, USB and RTCC working.

I also have to make a new PCB for the logger, something more USB-key shaped, with options for a rechargable battery. It should be my next blog post.

Stay tuned and as always, don’t hesitate to report any error.

Advertisements

Designing a simple and cheap temperature logger. Part 3: Open collector, FATs and de-debugging.

I had at last some time to work on the project. The prototype PCB is (at least, electrically) functional.

I started with making the TMP102 temperature sensor work. Nothing to say in particular, it’s just an other I2C-small-package-sensor. Or rather SMBus, but apart from the minimum speed clock requirements on the SMBus, the two protocols are alike.

On boot-up, the sensor gets configured and put to shutdown mode. Then, I use the “One Shot conversion” feature. When the right command is sent, the sensor starts a one-time temperature acquisition and returns to shutdown state, allowing to save as much as energy as possible.

The precision of the sensor is typically 0.5°C over its standard temperature range (-25°C to +85°C, the one I’ll be using) and the resolution is 0,0625°C (the LSB).

The temperature results are read on two bytes (12bits resolution). The first byte is the number before the comma, 2’s complemented for negative temperatures. The second byte contains the number after the comma. It has to be multiplied by 0.0625°C to get the real number.

I thought that a two digits after the comma precision should be enough (ie. by rounding), and instead of wasting CPU time with multiplication, I just made a string array containing all the results:

Temp_Precision[34]=”00061319253138445056636975818894″

Then, when the second temperature byte is read (Temp1byte), I just have to index the array with it (after a 4-bit shift, because it comes left-aligned). “TempASCII” is a string containing the temperature value:

TempASCII[i] = ',';
TempASCII[i+1] = Temp_Precision[Temp1byte*2];
TempASCII[i+2] = Temp_Precision[Temp1byte*2+1];

Which gives the following numbers vs. the original 0,0625-stepped ones, in °C:

The array could of course contain all the non-rounded numbers:

Temp_Precision[64]="0000062512501875250031253750437550005625625068757500812587509375"
TempASCII[i] = ',';
TempASCII[i+1] = Temp_Precision[Temp1byte*4];
TempASCII[i+2] = Temp_Precision[Temp1byte*4+1];
TempASCII[i+3] = Temp_Precision[Temp1byte*4+2];
TempASCII[i+4] = Temp_Precision[Temp1byte*4+3];

And so on. Useful trick instead of making a 0,0625 multiplication.

As said in earlier posts, I want to store the temperature readings in an Atmel AT25DF081 EEPROM. I chose it because it has a big capacity (8Mb) and a really low voltage supply value (1,8V). It connects via SPI to the microcontroller.

Now, with the great help of Microchip and its PICs I/Os being configurated as “Analog inputs” on startup and read as “0” when one forgets to set the right registers (ADCON0 & ADCON1), I had the great chance to spend two days measuring the EEPROM SPI link. Who would know the Analog ports “ANx” were also on the PORTB, and not only on the PORTA with ‘A’, as anolog? I know, RTFM.

During my investigations of the now perfectly working SPI, I could notice that the serial output pin of the memory (SDO) wasn’t an open collector, as I thought. It’s a pity it’s written nowhere in the Atmel’s datasheet. The Maximum input voltage of “3,8V”, Voh min of “Vcc-0,2” and no value for Voh max leave some room for imagination.

Well, the PIC seems to recognize the 1,8V SDO high state as high, even at higher speed, so it’s not such a big concern. A diode between the pin and the PIC will increase the EEPROM Voh to approx 2,5V (even if it also increases the EEPROM Vol of 0,6V). It could be an optional component on the final PCB design, replaced by a 0 Ohm resistor if the tests show no logical state decoding problem.

On the software page:

I took an example app from the Microchip’s USB libraries called “USB SD Card data logger”. It’s supposed to be an application for a logger with a SD card, being able to write on the card a value from a potentiometer.

It’s using the files SD-SPI.c and .h to access the SD Card, part of the Microchip’s MDD Filesystem. I started with these files and re-wrote them to make the project work with my EEPROM. Of course, that’s where the problems started..

On the filesystem, the default sector size is 512 bytes. As the PIC has to buffer the sector it’s writing to or reading from, there must be the same amount of memory allocated to the buffer as the size of the sector.  With the PIC18F26J50 I’m using, it cannot be bigger as 512 bytes.

The AT25DF081 EEPROM is organized into 256 bytes program pages. As they are smaller than the sector, it’s not a problem (I’m just making an address shift when the PIC wants to write more than 256 bytes). Reading is even easier, as there’s no page limit (it can be read starting form 0x000000 till 0x0FFFFF, the address counter on the EEPROM increments itself).

It gets complicated when you notice that the EEPROM’s erasing granularity doesn’t match the writing one. The smallest block to be erased is 4KB (then, 32KB, 64KB and 1MB) which means that to change even only one bit, all the 4KB must be read, erased and re-programmed.

AT25DF081 Memory space diagram

For now, I’m using only the first 512 Bytes of each 4KB page, but it limits the available space to 256 sectors of 512 bytes. That’s 128KB. Minus the space required for the filesystem and the sector for the logger configuration file, that leaves roughly 120KB of storage for the temperature measurements. Each of them will be something like “12/12/2011;23:59:59;127,00←↓” which is 28 Bytes. Divide 120KB by 28B, you get more or less a 40 000 temperature-acquisitions-worth-logger. Could be better and the idea of wasting 90% of my EEPROM is bad.

So, what exactly do we have to store in the EEPROM?

  • The FAT12 or 16 filesystem (3 or 4 sectors)
  • The configuration file (1 or 2 sector)
  • The temperature data

The FAT and the configuration file must be accessed often and must be user, computer-OS and PIC modifiable. But, the temperature data will only be written directly by the PIC and then only read by the user when the logging is over. The only modifications they (or rather the file containing them) will be subject to, are adding new measurements or erasing all the data for a new logging round. All that done by the PIC, which has direct access the the EEPROM.

I’ll need to modify the Microchip’s MMD filesystem routines to be able to directly write into a reserved block for the temp. measurements on the EEPROM. Plus, make the appropriate modifications in the FAT to allow a computer OS to read them, without being able to modify anything on the reserved block.

Quite a piece of work, especially because the library linking the USB procedures to the disk is complicated. And with optimized C code, it’s quite hard to debug the program, as the code pointer is jumping around all the time (I even thought it was a problem with my debugger/compiler/program/stack)

That’s for the next episode. I’ll also try to post the two files (SD-SPI.c and .h) I modified to use the AT25DF081 with the Microchip’s MDD and I’ll say a few words about the ICD3 debugger.

Designing a simple and cheap temperature logger. Part 2

Today, we’ll see how much the logger costs and try to approximate a power budget.

First, I was really surprised of how many people were interested in this project. I guess I have to keep on working on it.

I also forgot in my specifications that the logger shouldn’t need any driver or software to operate. An important plus for the product.

I took the standard prices form Farnell.fr to have an idea of the cost of a few prototypes. It should be possible to find a cheaper place or to order some of the components directly from the manufacturer (like the microcontroller). Let’s say it’s just to have a rough idea:

USB Temp Logger Components Prices

The final price is without VAT and shipping. For the PCB, I took iTeadStudio with two temperature loggers per 5x5cm board.

Then, I checked the cost for 1000 quantity for each component and the direct-from-manufacturer prices when possible. Once again, without VAT and shipping:

USB Temp Logger Components Prices 1000 qty

As expected, it’s twice cheaper and it’s only for 1000 pieces and without bulk factory prices. Of course, it’s based on the prototype and a final product would also have a casing and some ESD/CEM protections, which adds cost. But so far, I find it cheap enough.

For the power consumption part. The only thing I can do for now is to take values from the datasheets and try to calculate/extrapolate a theoretical life on battery power.

USB Temp Logger Power Chart

The logger will be designed to be in sleep mode most of the time. The microcontroller has a “deep sleep” mode (500nA) and wakes up by setting an alarm in the real time clock module. The temperature sensor and memory also have sleep functionalities.

Depending if the pins used for the serial buses are open drain or not, there can be current flowing trough the pull-up resistors, which could increase the sleep current as could also the 1,8V regulator’s quiescent current. Adding the numbers up gives an expected sleep time of almost two years (15,5uA) with one CR2032 battery. Not bad.

The power calculation gets more speculative for the temperature acquisition. The biggest problem will be to provide the flash memory current during writes. It’s 12mA. The time needed to program one byte is 15us. There will probably be a minimum of twenty bytes to be written, which gives 300us to write one temperature point, plus 25us for the deep sleep exit time. The PIC needs 1,5ms to exit the deep sleep mode. One temperature acquisition is performed in 35ms max. Let’s base our calculation on a 100ms not-sleep time (3mA) and a 400us flash write time (14mA).

We will also suppose the battery’s voltage won’t drop too much while supplying 12mA@1.8V (1.8V plus the regulator’s drop out voltage).

To summarise (currents and times not to scale):

USB Temp Logger Current Graph

Say the temperature measurement period would be one minute (60 per hour), we need to calculate the average current per hour and divide the battery capacity (225mAh) by this result. I keep times in milliseconds, thus the 3600 000 number (milliseconds in one hour). We will use the average current during wake-on: (3mA*100ms+15mA*0.4ms)/(100ms+0.4ms) = 3.04 mA

I’m using this formula (this site has a calculator: http://oregonembedded.com/batterycalc.htm):

Battery Life Equation

(Where BATTmAh is the battery capacity, ONmA is the current consumed during wake-on time, ONms is the temperature measurement time, ONph is the number of measurements per hour and OFFmA is the current in sleep mode. The result, th, is the time the battery will be able to power the logger, in hours. Divide by 24 to have the numbre of days)

{225 / [ ( 3.04*100.4*60) + (0.0155* ( 3600000 – ( 100.4*60) ) )]/3600000}/24

≈{225/ [(18313+55707)/3600000]}/24

≈{225/0.0205}/24

≡456 days.

Hmm.. That’s a lot and I don’t think it’s true. First, we don’t take account of the battery’s voltage evolution vs. minimum voltage required to power the components (remember, 2.5V). Then, I’m sure drawing 15mA from the battery, even for a very short period of time, will affect a lot its life time.

Let’s try to be more realistic and use different numbers.

Half of the battery capacity: 110mAh

More current drawn during on-time: 6mA

That’s 180 days, a bit more realistic.

I think the only way to get a true estimation of battery life during temperature acquisition operation will be to measure the real current for the wake-on time, plus measure the battery’s voltage drop while drawing the 15mA for memory write. But that’s for the next episode…

Designing a simple and cheap temperature logger. Part 1

[I’m writing about this project at the same time I’m working on it. It’s not finished yet, so please forgive the errors I could make (and don’t hesitate to slap them on my face)]

Some time ago, I was looking for a temperature logger. After checking what was available on the market and the prices, I decided to make one myself.

Specifications:

  • Battery powered
  • USB
  • No docking station
  • Configurable period of acquisition
  • Time stamped temperature
  • Cheap
  • At least 30 000 temperature acquisitions

Ideally, it would be USB key shaped. The presence of a battery and no docking station limit the temperature range to the battery maximum temperature specifications (-20°C -> 70°C).

As I didn’t find any easily available rechargeable battery that could fit into a USB stick enclosure, I decided to use a coin cell battery, CR2030 type. It’s 3V, 200mAh and should last enough to log the number of temperature measurements in the spec.

The temperature sensor would be a cheap serial one (they usually have a good precision). The memory, serial, cheap and big enough. Once plugged into an USB port, the logger would be seen as a USB key with two files. One with the temperature measurements and the other, user editable, with the configuration of the delay between every temperature acquisition. The acquisition will be started by a push on a microswitch.

To reduce the costs and size, the microcontroller should be able to manage the memory, the temperature sensor, the USB communication and have a real time clock.

Selected components:

  • Temp sensor: Texas Instr. TMP102AIDRLT
  • Microcontroller: PIC18F26J50
  • Memory: Atmel, AT25DF081-SSHN-B

The temperature sensor is 0,5°C accurate and takes from 1.4V to 3.6V for power supply with a 10µA current (1µA in sleep mode).

The memory has needs a supply power form 1.65V to 1.95V, 12mA write current and 8µA deep sleep mode current.

For the PIC, it’s 2.15V minimum for an average current of a few mA when running and a few µA in sleep mode (or nA in deep sleep mode)

Then, we need a voltage regulator for the PIC and temperature sensor (3.3V, when in USB mode) and an other one for the memory (1.8V).

When not plugged in a USB port, the coin cell powers directly the PIC and sensor but the memory is powered by its own regulator.

USB Temperature Logger Diagram

The power supply voltage can drop as low as 2,15 V (PIC’s minimum). With a new coin cell and a 0,35V Schottky diode (to prevent the USB current flowing back to the battery) the logger will stop working once the battery reaches 2,5V. This 0,5V margin may appear  small, but the battery’s standard discharge curve in the datasheet shows a fast drop in the remaining capacity, around 2,5V:

Coin Cell Discharge Curve 12k

It’s even more obvious with a lower current draw:

Coin Cell Discharge Curve 60k

After 2,5V, the remaining capacity can be as low as 10% of the initial capacity for high current discharge and even a lower percentage with a lower discharge current.

To be continued…

Part two is here

USB Temperature Logger 3D View