Simple and cheap temperature logger: v2.1 [UPDATE: EEPROM in BOM corrected (SST25WF080) ]

Hi everyone!

It’s been too long since the last update. But it’s there! The last version of the temperature logger, the hardware and the source code.

USB-Temperature-Logger---Component-side

USB-Temperature-Logger---Battery-side

The changes since the last version:

Hardware

  • New footprint for the PIC18F26J50, easier to solder.
  • Only 0603 or bigger resistors/capacitors.
  • The serial EEPROM is now a SST25WF080, still 8Mb, but easier to source.
  • Some components price-optimized (USB connector, switch, 3.3V reg).

Software

  • Each logger can get a number, shown in the mass storage device drive name: simply add an asterisk (*) followed by the number you want to assign to the logger (between 0 and 65535) in the config.txt file (after the logging period), save the file and format the logger.
  • Bug fixes in the FAT12 functions, but the logging space is still limited to the half of the EEPROM.
  • Other bug fixes (month and year change, added robustness). The memory and the EEPROM are now scanned when you plug the logger back, to get the last data and reconstruct the filesystem in the case of a battery failure.

Temperature Logger 2.1: Schematics and basic BOM

The Altium files

The source code (based on Microchip Applications libraries -Device – Mass Storage – SD Card data logger- MPLABX)

I also made a program/debug adapter (especially useful for debugging):

USB-Temperature-Logger-Programming-tool

USB-Temperature-Logger-Programming-tool1This is the “office” version of my previous peg-adapter 😉

A lot of people were asking me if I was selling these loggers. Unfortunately, no. The design and functionalities are sexy, but I don’t have the capacity to launch a production on my own. And if I had do make only a small batch, the price would be totally uncompetitive.

A big thanks for all the people who donated a little something to motivate me!! (or to help me be more ashamed of the lack of updates)

All the contents (except the parts of Microchip’s code in the source) are under the Creative commons license, Attribution – Share Alike – Non Commercial.

Creative Commons License
Simple temperature logger: v2.1 by Jean Wlodarski is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Simple and cheap temperature logger: New update coming

I’m working on a new revision of the temperature logger: new PCB and new firmware. It’s easier to solder, as the resistors/caps are now 0603 instead of 0402 and the PIC package is now a SSOP28 (instead of QFN).
The sleep current is smaller than 10uA and below 1mA when measuring the temperature (plus a few mA every 20-or-so logs, when writing to EEPROM).

The EEPROM is a SST25WF080 (still 1,8V 8Mb), as the previous Atmel memory is not available any more.

Some 3D renderings of the PCB:

Simple and cheap temperature logger 2.1

The USB connector is a Multicomp MC32604, cheaper than the previous MCKUSBX-SMT2AP1S-W30. (I couldn’t find a suitable 3D model for the PCB rendering)

The ISCP connector for programming the firmware  is SMT but fits the Microchip’s ICD header:

The new firmware has functions for the new EEPROM, a low battery indication, more robust logging and USB update capabilities. Plus a lot of bugfixes and optimizations.

Now, to complete the firmware, I have to build at least one logger, which means ordering the PCB and the components. That’s why I added a “donate” button to this blog (on the top right side of the page, just under the title), especially I don’t earn any money with this blog and content is free (and will stay free, of course).

If my work helped you, if you find it interesting, if you want more articles and more projects, dropping a few coins will motivate me and help me to materialize all the projects waiting in my head. 🙂

Stay tuned for more!

Jean.

Designing a simple and cheap temperature logger. Part 7: Schematics, PCB and source code.

I decided to publish all this project under Creative Commons license (Attribution, Derivatives, Non commercial, ShareAlike). The source code contains a FAT12 filesystem that can be reused to make custom flash drives for other projects.

USB Temperature Logger

(PCB version 1.2, hence the misaligned 3.3V regulator to include a diode. Corrected in the V2.0)

USB Temperature Logger 2

USB Temp Logger Schematics

You’ll find the schematics here: Simple USB temperature logger schematics

The Altium files (including a routed PCB) there: Simple USB temperature logger Altium files

The source code (based on Microchip Applications libraries -Device – Mass Storage – SD Card data logger- MPLABX): Simple USB temperature logger source code

The source code could be improved to increase the robustness of the logger (especially in the case of battery power failure while on logging mode). The PCB allows the monitoring of the battery voltage, it just needs to be implemented in the firmware.

I hope my work will be useful to someone!

Don’t hesitate to contact me (mail in the About section) for questions or comments!

Designing a simple and cheap temperature logger. Part 4: The junk in the trunk.

I somehow managed to solve the problem of the serial flash memory write block size vs. the erase block size. I just had to write a custom FAT12 filesystem..

As seen before, the Temperature Logger will act like an USB memory stick. There will be at least two files. One with the time stamped temperature measurements and one file allowing the user to set the logger’s configuration.

The configuration file must be accessible by any OS and modifiable. It will configure the temperature measurement period, from one every second till one every 24h (86400 seconds), in seconds. It’s also used to synchronize the PIC’s real time clock and calendar to the computer’s current time and date.

So far, it looks like that:

#THIS IS THE CONFIGURATION FILE
Setting the temperature logging period, in seconds
1<period<86400
Don't forget to add a "#" before the number:
#70
#END OF THE CONFIGURATION FILE

Pretty simple. When it’s saved, the PIC will read the number following the “#” and the time and date of the file modification.

The FAT filesystem is divided into five sections, each of them divided into sectors:

  • The Master Boot Record
  • The Boot Record
  • The FAT (doubled)
  • The Directory
  • The Data

When a file is written, its content will be placed in the Data section. The name and other information will be in the Directory. The FAT will then describe where the different data pieces of the file are located on the disk.

I took the standard sector size of 512 bytes (the size of the PIC’s buffer and the double of the EEPROM’s write block).

The MBR, the Boot Sector and the Directory are 512B from the OS point of view and 4KB on the EEPROM. I need five FAT pages to map the capacity of my EEPROM. Here’s a map of my filesystem:

FAT12 MicrochipAll the sectors occupied by the filesystem sections are 4K wide.

(The help.txt file contains the basic explanations on how to start/stop temperature acquisition and how to erase the data.csv file)

I modified the Microchip’s SD-SPI.c file to be able to talk to my memory, instead of a SD card. The main functions are:

BYTE MDD_SDSPI_SectorRead(DWORD sector_addr, BYTE* buffer)
BYTE MDD_SDSPI_SectorWrite(DWORD sector_addr, BYTE* buffer, BYTE allowWriteToZero)

They both take the address of the EEPROM sector to be written to, in LBA format (ie. the sector numbers) and translate them to the real EEPROM address. The buffer is the 512B PIC’s memory zone to write to the EEPROM or to fill with the EEPROM data.

An if statement chooses the address shift. Everything that’s below the DATA.CSV file will be 12-bits shifted ($02 becomes $2000) and the 4KB zone will be erased before any write occurs. If the write or read is in the zone where the temperature measurements are, the shift will be relative to the DATA.CSV address ($2E000), and looks like:

Address = Address - $2E
Address = Address << 9
Address = Address + $2E000

Which means, for example, that if we read or write the third sector of DATA.CSV, the function will be called with $30 and will result in the EEPROM address of $2E400, which is the third sector of the EEPROM after $2E000.

Of course, any write to this zone by the OS has to be forbidden.

For this, the Microchip USB Mass Storage Device driver uses the same functions as above, but in the usb_function_msd.c file. It uses an other buffer, msd_buffer, 512B too.

Every time the computer OS wants to read or write into the EEPROM, the data is cached in this buffer. Two functions are doing that:

BYTE MSDReadHandler(void)

BYTE MSDWriteHandler(void)

In each of these functions, I inserted a test to know what and where the OS wants to read or write.

In the case of a read, I’m not doing a lot for now (the MDD_SDSPI_SectorRead function already handles it).

For the writes, I first test if the OS is writing the data of config.txt file. If yes, I search for the period string and write the data to the memory. If the OS writes the File Dierctory, I read out the new date of the config.txt file but I write the original contents of the File Directory in the EEPROM instead of the new ones.

For any other write from the OS, I just answer the write was OK, but don’t write the actual data to the EEPROM.

This way, I’m sure the OS won’t fill the memory with anything else but the right data. It’s especially true for OSX, who wants to write Spotlight, Trash and info files in any USB stick it finds.

Here’s a simplified diagram of the modified part of SectorWrite function:

  • if msd_buffer contains the config.txt start and end string -> Extract the period, write the buffer to the config.txt file address.
  • if msd_buffer contains the File Directory -> Extract the time and date from config.txt, if it’s newer than the original date. Erase the msd_buffer and fill it with the original contents, minus the date and time fields. Write the buffer to the File Directory address.
  • in any other case, perform a dummy write.

That way, we save a lot of space in the EEPROM. If it would have been divided into 4KB sectors, with only the first 512B usable, we would have ended with a ≈100KB disk ((($FFFFF-$2E000)/$1000)*$200). Now, even if some space is wasted for the filesystem, we have a ≈840KB drive ($FFFFF-$2E000). It means something like 30 000 timestamped temperature measurements.

Some precisions about the FAT:

The FAT is used to keep track of every file on the disk. Each of the FAT entries matches a sector on the disk (that’s why my FAT is 5 sectors long, we need 5*(512/1,5) bytes to map our 1648 sectors). Here’s my FAT12 capture, as it comes after initialization:

FAT - FAT12 - Temperature Logger - FormatedThe first two entries are sector 0 and sector 1. They’re reserved. That’s why the data zone is located starting $2C000 (each sector is $1000 bytes before the DATA.CSV file and the FAT ends at $0C000).

The three next entries are $FFF,$FFF and $FFF which means that the sector contains one complete file (config.txt, help.txt and data.csv). (It’s easy to understand why small files are wasting space: Every file has to occupy at least one sector, even if it’s smaller than the size of the sector)

When a file is spread on more than one sector, its FAT entry will have the next sector address.

Unfortunately, the entries are coded in 12 bits values, and in big-endian format. If we want to write xyz for the sector 4 and XYZ for the sector 5, we have to write:

Yes, could be easier (it actually is, with FAT16 and FAT32, but our memory is too small to use those formats)

Here’s an example with the data.csv file spreading on 15 sectors (512*15 Bytes wide):

FAT12 - 19 sectors

If we decode the 12-bit addresses, it gives:

FAT12 - Decoded

There is one file in sector 2. One in sector 3. The third file starts on sector 4 and goes till sector 18.

I made a function to add entries every time a buffer full of temperature logs is written into the EEPROM. The 12-bit address management looks like that:

if(FatSector & 1) //Odd Sector Number
 {
 FATEntryPtr = FatSector-1;
 FATEntryPtr = FATEntryPtr + (FATEntryPtr>>1); // *1,5
 FatDWordA.w[0] = ( (FatSector+1) & 0xFFF);
 FatDWordA.w[1] = ( (FatSector) & 0xFFF);
 //We have:  0x yz 0X YZ and we want:
 //          00 yz Zx XY
 FatDWordB.v[0] = ((FatDWordA.v[1]&0x0F)<<4) | ((FatDWordA.v[0]&0xF0)>>4);
 FatDWordB.v[1] = ((FatDWordA.v[3]&0x0F)) | ((FatDWordA.v[0]&0x0F)<<4);
 FatDWordB.v[2] = FatDWordA.v[2];
 //The FatDWordB Contains what we want to write to the FAT, for the current //Sector and the next one.
 msd_buffer[FATEntryPtr] = FatDWordB.v[2];
 msd_buffer[FATEntryPtr+1] = FatDWordB.v[1];
 msd_buffer[FATEntryPtr+2] = FatDWordB.v[0];
 msd_buffer[FATEntryPtr+3] = 0xFF; //And write the end of the file for the //next sector entry
 msd_buffer[FATEntryPtr+4] = 0x0F;
 }
else //Even Number
 {
 FATEntryPtr = FatSector;
 FATEntryPtr = FATEntryPtr + (FATEntryPtr>>1); // *1,5
 //We want to write the next sector address in the current one, plus the //end of the file on the next sector
 FatDWordA.w[0] = 0x0FFF;    //End of the file for the next sector
 FatDWordA.w[1] = ( (FatSector+1) & 0xFFF); //In the current sector: the next //sector address.
 //We have:  0x yz 0X YZ and we want:
 //          00 yz Zx XY
 FatDWordB.v[0] = ((FatDWordA.v[1]&0x0F)<<4) | ((FatDWordA.v[0]&0xF0)>>4);
 FatDWordB.v[1] = ((FatDWordA.v[3]&0x0F)) | ((FatDWordA.v[0]&0x0F)<<4);
 FatDWordB.v[2] = FatDWordA.v[2];
 //The FatDWordB Contains what we want to write to the FAT, for the current //Sector and the next one.
 msd_buffer[FATEntryPtr] = FatDWordB.v[2];
 msd_buffer[FATEntryPtr+1] = FatDWordB.v[1];
 msd_buffer[FATEntryPtr+2] = FatDWordB.v[0];
 }

I’m using the msd_buffer to save the FAT during temperature acquisition (and the gDataBuffer to save the temperature logs) between two writes to the EEPROM, when on battery power.

Every time a gDataBuffer is full, it’s written to the EEPROM in the data zone and a new FAT entry is added to the msd_buffer. Once this one is full, it’s written to the EEPROM at the FAT address.

The variable FatSector contains the current sector in the FAT. FatDWordA and FatDWordB are temporary Double-Word variables used to compute the three-bytes-12bits FAT entries.

This is MCC18 PIC18 code, based on Microchip USB and MSD libraries. I wanted to share this snippet, as I couldn’t find any good PIC code example on the internet. I’m sure it could be optimized and simplified, but like that, it’s maybe easier to understand how it works.

I think it’s all for now. My next step will be to consolidate my program and code the power-managed temperature acquisition. I already have the Temp. sensor, EEPROM, USB and RTCC working.

I also have to make a new PCB for the logger, something more USB-key shaped, with options for a rechargable battery. It should be my next blog post.

Stay tuned and as always, don’t hesitate to report any error.

Designing a simple and cheap temperature logger. Part 3: Open collector, FATs and de-debugging.

I had at last some time to work on the project. The prototype PCB is (at least, electrically) functional.

I started with making the TMP102 temperature sensor work. Nothing to say in particular, it’s just an other I2C-small-package-sensor. Or rather SMBus, but apart from the minimum speed clock requirements on the SMBus, the two protocols are alike.

On boot-up, the sensor gets configured and put to shutdown mode. Then, I use the “One Shot conversion” feature. When the right command is sent, the sensor starts a one-time temperature acquisition and returns to shutdown state, allowing to save as much as energy as possible.

The precision of the sensor is typically 0.5°C over its standard temperature range (-25°C to +85°C, the one I’ll be using) and the resolution is 0,0625°C (the LSB).

The temperature results are read on two bytes (12bits resolution). The first byte is the number before the comma, 2’s complemented for negative temperatures. The second byte contains the number after the comma. It has to be multiplied by 0.0625°C to get the real number.

I thought that a two digits after the comma precision should be enough (ie. by rounding), and instead of wasting CPU time with multiplication, I just made a string array containing all the results:

Temp_Precision[34]=”00061319253138445056636975818894″

Then, when the second temperature byte is read (Temp1byte), I just have to index the array with it (after a 4-bit shift, because it comes left-aligned). “TempASCII” is a string containing the temperature value:

TempASCII[i] = ',';
TempASCII[i+1] = Temp_Precision[Temp1byte*2];
TempASCII[i+2] = Temp_Precision[Temp1byte*2+1];

Which gives the following numbers vs. the original 0,0625-stepped ones, in °C:

The array could of course contain all the non-rounded numbers:

Temp_Precision[64]="0000062512501875250031253750437550005625625068757500812587509375"
TempASCII[i] = ',';
TempASCII[i+1] = Temp_Precision[Temp1byte*4];
TempASCII[i+2] = Temp_Precision[Temp1byte*4+1];
TempASCII[i+3] = Temp_Precision[Temp1byte*4+2];
TempASCII[i+4] = Temp_Precision[Temp1byte*4+3];

And so on. Useful trick instead of making a 0,0625 multiplication.

As said in earlier posts, I want to store the temperature readings in an Atmel AT25DF081 EEPROM. I chose it because it has a big capacity (8Mb) and a really low voltage supply value (1,8V). It connects via SPI to the microcontroller.

Now, with the great help of Microchip and its PICs I/Os being configurated as “Analog inputs” on startup and read as “0” when one forgets to set the right registers (ADCON0 & ADCON1), I had the great chance to spend two days measuring the EEPROM SPI link. Who would know the Analog ports “ANx” were also on the PORTB, and not only on the PORTA with ‘A’, as anolog? I know, RTFM.

During my investigations of the now perfectly working SPI, I could notice that the serial output pin of the memory (SDO) wasn’t an open collector, as I thought. It’s a pity it’s written nowhere in the Atmel’s datasheet. The Maximum input voltage of “3,8V”, Voh min of “Vcc-0,2” and no value for Voh max leave some room for imagination.

Well, the PIC seems to recognize the 1,8V SDO high state as high, even at higher speed, so it’s not such a big concern. A diode between the pin and the PIC will increase the EEPROM Voh to approx 2,5V (even if it also increases the EEPROM Vol of 0,6V). It could be an optional component on the final PCB design, replaced by a 0 Ohm resistor if the tests show no logical state decoding problem.

On the software page:

I took an example app from the Microchip’s USB libraries called “USB SD Card data logger”. It’s supposed to be an application for a logger with a SD card, being able to write on the card a value from a potentiometer.

It’s using the files SD-SPI.c and .h to access the SD Card, part of the Microchip’s MDD Filesystem. I started with these files and re-wrote them to make the project work with my EEPROM. Of course, that’s where the problems started..

On the filesystem, the default sector size is 512 bytes. As the PIC has to buffer the sector it’s writing to or reading from, there must be the same amount of memory allocated to the buffer as the size of the sector.  With the PIC18F26J50 I’m using, it cannot be bigger as 512 bytes.

The AT25DF081 EEPROM is organized into 256 bytes program pages. As they are smaller than the sector, it’s not a problem (I’m just making an address shift when the PIC wants to write more than 256 bytes). Reading is even easier, as there’s no page limit (it can be read starting form 0x000000 till 0x0FFFFF, the address counter on the EEPROM increments itself).

It gets complicated when you notice that the EEPROM’s erasing granularity doesn’t match the writing one. The smallest block to be erased is 4KB (then, 32KB, 64KB and 1MB) which means that to change even only one bit, all the 4KB must be read, erased and re-programmed.

AT25DF081 Memory space diagram

For now, I’m using only the first 512 Bytes of each 4KB page, but it limits the available space to 256 sectors of 512 bytes. That’s 128KB. Minus the space required for the filesystem and the sector for the logger configuration file, that leaves roughly 120KB of storage for the temperature measurements. Each of them will be something like “12/12/2011;23:59:59;127,00←↓” which is 28 Bytes. Divide 120KB by 28B, you get more or less a 40 000 temperature-acquisitions-worth-logger. Could be better and the idea of wasting 90% of my EEPROM is bad.

So, what exactly do we have to store in the EEPROM?

  • The FAT12 or 16 filesystem (3 or 4 sectors)
  • The configuration file (1 or 2 sector)
  • The temperature data

The FAT and the configuration file must be accessed often and must be user, computer-OS and PIC modifiable. But, the temperature data will only be written directly by the PIC and then only read by the user when the logging is over. The only modifications they (or rather the file containing them) will be subject to, are adding new measurements or erasing all the data for a new logging round. All that done by the PIC, which has direct access the the EEPROM.

I’ll need to modify the Microchip’s MMD filesystem routines to be able to directly write into a reserved block for the temp. measurements on the EEPROM. Plus, make the appropriate modifications in the FAT to allow a computer OS to read them, without being able to modify anything on the reserved block.

Quite a piece of work, especially because the library linking the USB procedures to the disk is complicated. And with optimized C code, it’s quite hard to debug the program, as the code pointer is jumping around all the time (I even thought it was a problem with my debugger/compiler/program/stack)

That’s for the next episode. I’ll also try to post the two files (SD-SPI.c and .h) I modified to use the AT25DF081 with the Microchip’s MDD and I’ll say a few words about the ICD3 debugger.