I’ve found a couple of links you might find helpful. Take a look at http://stackoverflow.com/questions/14820004/bluetooth-pairing-in-c-bluez-on-linux . There you’ll find a link to a Bluetooth book with several examples in c, and configuration of setting up and pairing. Also, take a look at this document https://communities.intel.com/docs/DOC-23618 . It’s the Bluetooth guide of the Edison. Maybe you can find some useful information here.
You should perform following HCE commands:
1. LE Set Scan Parameters.
2. LE Set Event Mask ( add LE Advertising Report Event ).
3. LE Set Scan Enable
4. Wait for advertising reports ( EVT_LE_META_EVENT 0x3E -> EVT_LE_ADVERTISING_REPORT = 0x02 ).
RSSI value it's in a frame and can be parsed using source code of hci.c::evt_le_advertising_report_dump method.
UPDATE: Please check scan.c file -> glock45/intel-edison-playground · GitHub