240 lines
11 KiB
Plaintext
240 lines
11 KiB
Plaintext
|
* USING SPIFFS
|
||
|
|
|
||
|
|
TODO
|
||
|
|
|
||
|
|
|
||
|
|
* SPIFFS DESIGN
|
||
|
|
|
||
|
|
Spiffs is inspired by YAFFS. However, YAFFS is designed for NAND flashes, and
|
||
|
|
for bigger targets with much more ram. Nevertheless, many wise thoughts have
|
||
|
|
been borrowed from YAFFS when writing spiffs. Kudos!
|
||
|
|
|
||
|
|
The main complication writing spiffs was that it cannot be assumed the target
|
||
|
|
has a heap. Spiffs must go along only with the work ram buffer given to it.
|
||
|
|
This forces extra implementation on many areas of spiffs.
|
||
|
|
|
||
|
|
|
||
|
|
** SPI flash devices using NOR technology
|
||
|
|
|
||
|
|
Below is a small description of how SPI flashes work internally. This is to
|
||
|
|
give an understanding of the design choices made in spiffs.
|
||
|
|
|
||
|
|
SPI flash devices are physically divided in blocks. On some SPI flash devices,
|
||
|
|
blocks are further divided into sectors. Datasheets sometimes name blocks as
|
||
|
|
sectors and vice versa.
|
||
|
|
|
||
|
|
Common memory capacaties for SPI flashes are 512kB up to 8MB of data, where
|
||
|
|
blocks may be 64kB. Sectors can be e.g. 4kB, if supported. Many SPI flashes
|
||
|
|
have uniform block sizes, whereas others have non-uniform - the latter meaning
|
||
|
|
that e.g. the first 16 blocks are 4kB big, and the rest are 64kB.
|
||
|
|
|
||
|
|
The entire memory is linear and can be read and written in random access.
|
||
|
|
Erasing can only be done block- or sectorwise; or by mass erase.
|
||
|
|
|
||
|
|
SPI flashes can normally be erased from 100.000 up to 1.000.000 cycles before
|
||
|
|
they fail.
|
||
|
|
|
||
|
|
A clean SPI flash from factory have all bits in entire memory set to one. A
|
||
|
|
mass erase will reset the device to this state. Block or sector erasing will
|
||
|
|
put the all bits in the area given by the sector or block to ones. Writing to a
|
||
|
|
NOR flash pulls ones to zeroes. Writing 0xFF to an address is simply a no-op.
|
||
|
|
|
||
|
|
Writing 0b10101010 to a flash address holding 0b00001111 will yield 0b00001010.
|
||
|
|
|
||
|
|
This way of "write by nand" is used considerably in spiffs.
|
||
|
|
|
||
|
|
Common characteristics of NOR flashes are quick reads, but slow writes.
|
||
|
|
|
||
|
|
And finally, unlike NAND flashes, NOR flashes seem to not need any error
|
||
|
|
correction. They always write correctly I gather.
|
||
|
|
|
||
|
|
|
||
|
|
** Spiffs logical structure
|
||
|
|
|
||
|
|
Some terminology before proceeding. Physical blocks/sectors means sizes stated
|
||
|
|
in the datasheet. Logical blocks and pages is something the integrator choose.
|
||
|
|
|
||
|
|
|
||
|
|
** Blocks and pages
|
||
|
|
|
||
|
|
Spiffs is allocated to a part or all of the memory of the SPI flash device.
|
||
|
|
This area is divided into logical blocks, which in turn are divided into
|
||
|
|
logical pages. The boundary of a logical block must coincide with one or more
|
||
|
|
physical blocks. The sizes for logical blocks and logical pages always remain
|
||
|
|
the same, they are uniform.
|
||
|
|
|
||
|
|
Example: non-uniform flash mapped to spiffs with 128kB logical blocks
|
||
|
|
|
||
|
|
PHYSICAL FLASH BLOCKS SPIFFS LOGICAL BLOCKS: 128kB
|
||
|
|
|
||
|
|
+-----------------------+ - - - +-----------------------+
|
||
|
|
| Block 1 : 16kB | | Block 1 : 128kB |
|
||
|
|
+-----------------------+ | |
|
||
|
|
| Block 2 : 16kB | | |
|
||
|
|
+-----------------------+ | |
|
||
|
|
| Block 3 : 16kB | | |
|
||
|
|
+-----------------------+ | |
|
||
|
|
| Block 4 : 16kB | | |
|
||
|
|
+-----------------------+ | |
|
||
|
|
| Block 5 : 64kB | | |
|
||
|
|
+-----------------------+ - - - +-----------------------+
|
||
|
|
| Block 6 : 64kB | | Block 2 : 128kB |
|
||
|
|
+-----------------------+ | |
|
||
|
|
| Block 7 : 64kB | | |
|
||
|
|
+-----------------------+ - - - +-----------------------+
|
||
|
|
| Block 8 : 64kB | | Block 3 : 128kB |
|
||
|
|
+-----------------------+ | |
|
||
|
|
| Block 9 : 64kB | | |
|
||
|
|
+-----------------------+ - - - +-----------------------+
|
||
|
|
| ... | | ... |
|
||
|
|
|
||
|
|
A logical block is divided further into a number of logical pages. A page
|
||
|
|
defines the smallest data holding element known to spiffs. Hence, if a file
|
||
|
|
is created being one byte big, it will occupy one page for index and one page
|
||
|
|
for data - it will occupy 2 x size of a logical page on flash.
|
||
|
|
So it seems it is good to select a small page size.
|
||
|
|
|
||
|
|
Each page has a metadata header being normally 5 to 9 bytes. This said, a very
|
||
|
|
small page size will make metadata occupy a lot of the memory on the flash. A
|
||
|
|
page size of 64 bytes will waste 8-14% on metadata, while 256 bytes 2-4%.
|
||
|
|
So it seems it is good to select a big page size.
|
||
|
|
|
||
|
|
Also, spiffs uses a ram buffer being two times the page size. This ram buffer
|
||
|
|
is used for loading and manipulating pages, but it is also used for algorithms
|
||
|
|
to find free file ids, scanning the file system, etc. Having too small a page
|
||
|
|
size means less work buffer for spiffs, ending up in more reads operations and
|
||
|
|
eventually gives a slower file system.
|
||
|
|
|
||
|
|
Choosing the page size for the system involves many factors:
|
||
|
|
- How big is the logical block size
|
||
|
|
- What is the normal size of most files
|
||
|
|
- How much ram can be spent
|
||
|
|
- How much data (vs metadata) must be crammed into the file system
|
||
|
|
- How fast must spiffs be
|
||
|
|
- Other things impossible to find out
|
||
|
|
|
||
|
|
So, chosing the Optimal Page Size (tm) seems tricky, to say the least. Don't
|
||
|
|
fret - there is no optimal page size. This varies from how the target will use
|
||
|
|
spiffs. Use the golden rule:
|
||
|
|
|
||
|
|
~~~ Logical Page Size = Logical Block Size / 256 ~~~
|
||
|
|
|
||
|
|
This is a good starting point. The final page size can then be derived through
|
||
|
|
heuristical experimenting for us non-analytical minds.
|
||
|
|
|
||
|
|
|
||
|
|
** Objects, indices and look-ups
|
||
|
|
|
||
|
|
A file, or an object as called in spiffs, is identified by an object id.
|
||
|
|
Another YAFFS rip-off. This object id is a part of the page header. So, all
|
||
|
|
pages know to which object/file they belong - not counting the free pages.
|
||
|
|
|
||
|
|
An object is made up of two types of pages: object index pages and data pages.
|
||
|
|
Data pages contain the data written by user. Index pages contain metadata about
|
||
|
|
the object, more specifically what data pages are part of the object.
|
||
|
|
|
||
|
|
The page header also includes something called a span index. Let's say a file
|
||
|
|
is written covering three data pages. The first data page will then have span
|
||
|
|
index 0, the second span index 1, and the last data page will have span index
|
||
|
|
2. Simple as that.
|
||
|
|
|
||
|
|
Finally, each page header contain flags, telling if the page is used,
|
||
|
|
deleted, finalized, holds index or data, and more.
|
||
|
|
|
||
|
|
Object indices also have span indices, where an object index with span index 0
|
||
|
|
is referred to as the object index header. This page does not only contain
|
||
|
|
references to data pages, but also extra info such as object name, object size
|
||
|
|
in bytes, flags for file or directory, etc.
|
||
|
|
|
||
|
|
If one were to create a file covering three data pages, named e.g.
|
||
|
|
"spandex-joke.txt", given object id 12, it could look like this:
|
||
|
|
|
||
|
|
PAGE 0 <things to be unveiled soon>
|
||
|
|
|
||
|
|
PAGE 1 page header: [obj_id:12 span_ix:0 flags:USED|DATA]
|
||
|
|
<first data page of joke>
|
||
|
|
|
||
|
|
PAGE 2 page header: [obj_id:12 span_ix:1 flags:USED|DATA]
|
||
|
|
<second data page of joke>
|
||
|
|
|
||
|
|
PAGE 3 page header: [obj_id:545 span_ix:13 flags:USED|DATA]
|
||
|
|
<some data belonging to object 545, probably not very amusing>
|
||
|
|
|
||
|
|
PAGE 4 page header: [obj_id:12 span_ix:2 flags:USED|DATA]
|
||
|
|
<third data page of joke>
|
||
|
|
|
||
|
|
PAGE 5 page header: [obj_id:12 span_ix:0 flags:USED|INDEX]
|
||
|
|
obj ix header: [name:spandex-joke.txt size:600 bytes flags:FILE]
|
||
|
|
obj ix: [1 2 4]
|
||
|
|
|
||
|
|
Looking in detail at page 5, the object index header page, the object index
|
||
|
|
array refers to each data page in order, as mentioned before. The index of the
|
||
|
|
object index array correlates with the data page span index.
|
||
|
|
|
||
|
|
entry ix: 0 1 2
|
||
|
|
obj ix: [1 2 4]
|
||
|
|
| | |
|
||
|
|
PAGE 1, DATA, SPAN_IX 0 --------/ | |
|
||
|
|
PAGE 2, DATA, SPAN_IX 1 --------/ |
|
||
|
|
PAGE 4, DATA, SPAN_IX 2 --------/
|
||
|
|
|
||
|
|
Things to be unveiled in page 0 - well.. Spiffs is designed for systems low on
|
||
|
|
ram. We cannot keep a dynamic list on the whereabouts of each object index
|
||
|
|
header so we can find a file fast. There might not even be a heap! But, we do
|
||
|
|
not want to scan all page headers on the flash to find the object index header.
|
||
|
|
|
||
|
|
The first page(s) of each block contains the so called object look-up. These
|
||
|
|
are not normal pages, they do not have a header. Instead, they are arrays
|
||
|
|
pointing out what object-id the rest of all pages in the block belongs to.
|
||
|
|
|
||
|
|
By this look-up, only the first page(s) in each block must to scanned to find
|
||
|
|
the actual page which contains the object index header of the desired object.
|
||
|
|
|
||
|
|
The object lookup is redundant metadata. The assumption is that it presents
|
||
|
|
less overhead reading a full page of data to memory from each block and search
|
||
|
|
that, instead of reading a small amount of data from each page (i.e. the page
|
||
|
|
header) in all blocks. Each read operation from SPI flash normally contains
|
||
|
|
extra data as the read command itself and the flash address. Also, depending on
|
||
|
|
the underlying implementation, other criterions may need to be passed for each
|
||
|
|
read transaction, like mutexes and such.
|
||
|
|
|
||
|
|
The veiled example unveiled would look like this, with some extra pages:
|
||
|
|
|
||
|
|
PAGE 0 [ 12 12 545 12 12 34 34 4 0 0 0 0 ...]
|
||
|
|
PAGE 1 page header: [obj_id:12 span_ix:0 flags:USED|DATA] ...
|
||
|
|
PAGE 2 page header: [obj_id:12 span_ix:1 flags:USED|DATA] ...
|
||
|
|
PAGE 3 page header: [obj_id:545 span_ix:13 flags:USED|DATA] ...
|
||
|
|
PAGE 4 page header: [obj_id:12 span_ix:2 flags:USED|DATA] ...
|
||
|
|
PAGE 5 page header: [obj_id:12 span_ix:0 flags:USED|INDEX] ...
|
||
|
|
PAGE 6 page header: [obj_id:34 span_ix:0 flags:USED|DATA] ...
|
||
|
|
PAGE 7 page header: [obj_id:34 span_ix:1 flags:USED|DATA] ...
|
||
|
|
PAGE 8 page header: [obj_id:4 span_ix:1 flags:USED|INDEX] ...
|
||
|
|
PAGE 9 page header: [obj_id:23 span_ix:0 flags:DELETED|INDEX] ...
|
||
|
|
PAGE 10 page header: [obj_id:23 span_ix:0 flags:DELETED|DATA] ...
|
||
|
|
PAGE 11 page header: [obj_id:23 span_ix:1 flags:DELETED|DATA] ...
|
||
|
|
PAGE 12 page header: [obj_id:23 span_ix:2 flags:DELETED|DATA] ...
|
||
|
|
...
|
||
|
|
|
||
|
|
Ok, so why are page 9 to 12 marked as 0 when they belong to object id 23? These
|
||
|
|
pages are deleted, so this is marked both in page header flags and in the look
|
||
|
|
up. This is an example where spiffs uses NOR flashes "nand-way" of writing.
|
||
|
|
|
||
|
|
As a matter of fact, there are two object id's which are special:
|
||
|
|
|
||
|
|
obj id 0 (all bits zeroes) - indicates a deleted page in object look up
|
||
|
|
obj id 0xff.. (all bits ones) - indicates a free page in object look up
|
||
|
|
|
||
|
|
Actually, the object id's have another quirk: if the most significant bit is
|
||
|
|
set, this indicates an object index page. If the most significant bit is zero,
|
||
|
|
this indicates a data page. So to be fully correct, page 0 in above example
|
||
|
|
would look like this:
|
||
|
|
|
||
|
|
PAGE 0 [ 12 12 545 12 *12 34 34 *4 0 0 0 0 ...]
|
||
|
|
|
||
|
|
where the asterisk means the msb of the object id is set.
|
||
|
|
|
||
|
|
This is another way to speed up the searches when looking for object indices.
|
||
|
|
By looking on the object id's msb in the object lookup, it is also possible
|
||
|
|
to find out whether the page is an object index page or a data page.
|
||
|
|
|