Someone from Intel or Arduino can probably answer this much better than I can. There is a reasonable amount of documentation up on the web including FAQs and some new specifications on how libraries are to be setup. Example: https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification
Couple of issues here. How Arduino searches for Header files. For the most part all libraries that are used by a sketch must be called out in the main sketch, including those which other libraries use. The Arduino system, then tries to locate the appropriate library. I do not remember the exact sequence that it searches, but it is something like:
a) Maybe in sketch folder (may depend if #include is using <> or "" )
b) In your Arduino Sketch folder/libraries
Now I am a bit unsure in order
c) Libraries contained in the specific Hardware libraries folder
d) Arduino installation libraries folder
A couple of notes:
a) Not all libraries support all architectures: This is the case you are running into with SoftwareSerial - I don't believe a version has been implemented for the Edison (I could be wrong), which is why the system is not finding it. Note, trying to use an AVR version for this will not work as I am very sure SoftwareSerial uses very AVR specific code like interrupts, registers and the like.
b) There may be multiple implementations for the same library, which can lead to some issues. I believe the latest Arduino release 1.6.3 now will display a warning message when you do a compile and it finds multiple implementations (thanks to Paul of PJRC - Teensy)
Again except for the issues associated with which libraries have been ported to the Edison, none of the library searching stuff is specific to Intel.
Hope that helps
Thanks KurtE for a very detailed and thorough answer. Great information.