Ticket #117 (closed issue: fixed)

Opened 10 months ago

Last modified 10 months ago

DB Insertion Error - Possible Corrupt Capture Parsing?

Reported by: kindlund Assigned to: xkovah
Priority: high Milestone: 1.0
Component: HoneyClient::DB Version: 0.99
Severity: none Keywords: db, capture, realtime, change, insert, key_name
Cc:

Description

Matt/Xeno:

We were testing the trunk as it stands and we noticed this type of error:

2007-11-05 16:58:56  WARN [HoneyClient::Manager::runSession] (lib/HoneyClient/Manager.pm:735) - VM Compromised.  Last Resource (http://www.craigslist.org/about/best/bli/440353839.html)
Perl exited with active threads:
        0 running and unjoined
        1 finished and unjoined
        0 running and detached
2007-11-05 16:58:56  INFO [HoneyClient::Manager::runSession] (lib/HoneyClient/Manager.pm:746) - Database Insert last url successful
2007-11-05 16:58:56  INFO [HoneyClient::Manager::runSession] (lib/HoneyClient/Manager.pm:756) - Database Update Client fk in last url
2007-11-05 16:58:56  INFO [HoneyClient::Manager::runSession] (lib/HoneyClient/Manager.pm:759) - Inserting Fingerprint Into Database.
$VAR1 = {
          'md5_hash' => 'a9a8de56f60f456297454cd8e5490f81',
          'processes' => [
                           {
                             'pid' => '988',
                             'file_system' => [],
                             'registry' => [
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Performance Refresh',
                                               'time' => '2007-11-05 16:56:22.781',
                                               'event_type' => 'SetValueKey',
                                               'value' => '1',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\WBEM\\PROVIDERS\\Performance'
                                             }
                                           ],
                             'name' => 'C:\\WINDOWS\\system32\\svchost.exe'
                           },
                           {
                             'pid' => '680',
                             'parent_name' => 'C:\\WINDOWS\\system32\\svchost.exe',
                             'file_system' => [],
                             'registry' => [
                                             {
                                               'value_type' => 'REG_SZ',
                                               'value_name' => 'Updating',
                                               'time' => '2007-11-05 16:58:04.59',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'WmiApRpl',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Last Counter',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fda',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Last Help',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fdb',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'First Counter',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Last Counter',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'First Help',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Last Help',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Object List',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Library Validation Code',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Updating',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'First Counter',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'First Help',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Last Counter',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Last Help',
                                               'time' => '2007-11-05 16:58:04.247',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_SZ',
                                               'value_name' => 'Updating',
                                               'time' => '2007-11-05 16:58:04.278',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'WmiApRpl',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Last Counter',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fe8',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Last Help',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fe9',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Last Counter',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fe8',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'Last Help',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fe9',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'First Counter',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fdc',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_DWORD',
                                               'value_name' => 'First Help',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => 'fdd',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_SZ',
                                               'value_name' => 'Object List',
                                               'time' => '2007-11-05 16:58:06.325',
                                               'event_type' => 'SetValueKey',
                                               'value' => '4060 4066',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Disable Performance Counters',
                                               'time' => '2007-11-05 16:58:06.341',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SYSTEM\\ControlSet001\\Services\\WmiApRpl\\Performance'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'Updating',
                                               'time' => '2007-11-05 16:58:06.341',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Perflib'
                                             }
                                           ],
                             'name' => 'C:\\WINDOWS\\system32\\wbem\\wmiadap.exe',
                             'parent_pid' => '988',
                             'created_time' => '2007-11-05 16:57:23.428',
                             'terminated_time' => '2007-11-05 16:58:11.232'
                           },
                           {
                             'pid' => '744',
                             'file_system' => [],
                             'registry' => [
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29653422,LowDateTime=2875513088,Name="C:\\\\WINDOWS\\\\System32\\\\DRIVERS\\\\ACPI.sys[ACPIMOFResource]"',
                                               'time' => '2007-11-05 16:57:36.805',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => '\\REG'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29653447,LowDateTime=3591330688,Name="C:\\\\WINDOWS\\\\System32\\\\DRIVERS\\\\mssmbios.sys[MofResource]"',
                                               'time' => '2007-11-05 16:57:37.71',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => '\\REG'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29653446,LowDateTime=2806297984,Name="C:\\\\WINDOWS\\\\System32\\\\DRIVERS\\\\intelppm.sys[PROCESSORWMI]"',
                                               'time' => '2007-11-05 16:57:37.180',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => '\\REGIS'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29435636,LowDateTime=863039744,Name="C:\\\\WINDOWS\\\\System32\\\\DRIVERS\\\\pcntpci5.sys[NdisMofResource]"',
                                               'time' => '2007-11-05 16:57:38.649',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => '\\REGISTRY\\'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29653422,LowDateTime=1215513088,Name="C:\\\\WINDOWS\\\\System32\\\\DRIVERS\\\\ipnat.sys[IPNATMofResource]"',
                                               'time' => '2007-11-05 16:57:40.102',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => '\\REGISTR'
                                             },
                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29666671,LowDateTime=3283808384,Name="C:\\\\WINDOWS\\\\System32\\\\Drivers\\\\HTTP.sys[UlMofResource]"',
                                               'time' => '2007-11-05 16:57:40.946',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => ''
                                             }
                                           ],
                             'name' => 'C:\\WINDOWS\\system32\\wbem\\wmiprvse.exe'
                           }
                         ]
        };

2007-11-05 16:58:57 FATAL [HoneyClient::DB::new] (lib/HoneyClient/DB.pm:445) - HoneyClient::DB::Regkey->new(): Object missing required attribute(s): key_name.
Error: HoneyClient::DB::Regkey->new(): Object missing required attribute(s): key_name.\n at lib/HoneyClient/Manager.pm line 760

Take a look at that last registry entry:

                                             {
                                               'value_type' => 'REG_NONE',
                                               'value_name' => 'WMIBinaryMofResource.HighDateTime=29666671,LowDateTime=3283808384,Name="C:\\\\WINDOWS\\\\System32\\\\Drivers\\\\HTTP.sys[UlMofResource]"',
                                               'time' => '2007-11-05 16:57:40.946',
                                               'event_type' => 'DeleteValueKey',
                                               'value' => '',
                                               'key_name' => ''
                                             }

See how it has no 'key_name' ? Xeno, is that expected? Or is this a bug? If so, then we need to update the DB code to support this. Matt, it's possible that this bug has already been fixed, if so, then we need to merge your changes from whichever branch you're committing to… towards the trunk branch.

Lastly, Xeno, it looks like these changes in sum should be ignored by the Registry exclusion list. Could you find out why the current RegistryMonitor.exl is unable to identify and properly exclude these changes (in trunk)?

Thanks, Darien

Attachments

realtime-changes.txt (8.9 kB) - added by kindlund on 11/06/07 14:02:41.
Realtime Changes Output

Change History

11/05/07 18:16:34 changed by kindlund

  • cc changed from xkovah to mbriggs.
  • owner changed from mbriggs to xkovah.

After further analysis, it appears the base issue is a parsing error within the Capture code. Xeno, check out these entries:

"2007-11-05 18:08:43.875","process","created","976","C:\WINDOWS\system32\svchost.exe","332","C:\WINDOWS\system32\wbem\wm
iadap.exe"
"2007-11-05 18:08:51.15","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REG","WMIBinaryMofR
esource.HighDateTime=29653422,LowDateTime=2875513088,Name="C:\\WINDOWS\\System32\\DRIVERS\\ACPI.sys[ACPIMOFResource]"","
REG_NONE",""
"2007-11-05 18:08:51.219","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REG","WMIBinaryMof
Resource.HighDateTime=29653447,LowDateTime=3591330688,Name="C:\\WINDOWS\\System32\\DRIVERS\\mssmbios.sys[MofResource]"",
"REG_NONE",""
"2007-11-05 18:08:51.703","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REGISTRY\","WMIBin
aryMofResource.HighDateTime=29435636,LowDateTime=863039744,Name="C:\\WINDOWS\\System32\\DRIVERS\\pcntpci5.sys[NdisMofRes
ource]"","REG_NONE",""
"2007-11-05 18:08:51.828","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REGISTR","WMIBinar
yMofResource.HighDateTime=29653422,LowDateTime=1215513088,Name="C:\\WINDOWS\\System32\\DRIVERS\\ipnat.sys[IPNATMofResour
ce]"","REG_NONE",""
"2007-11-05 18:08:51.953","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","","WMIBinaryMofReso
urce.HighDateTime=29666671,LowDateTime=3283808384,Name="C:\\WINDOWS\\System32\\Drivers\\HTTP.sys[UlMofResource]"","REG_N
ONE",""

Notice the registry paths listed: "\REG", "\REGISTR", and "\REGISTRY\" … Those aren't valid registry paths (i.e., HKLM\\). So, could you please identify whether this data is intentional or unintentional and your recommendations on how to resolve it?

Thanks, — Darien

11/06/07 08:19:51 changed by xkovah

Is this a reproducible error, or was it found via a normal run?

Xeno

11/06/07 09:54:37 changed by xkovah

  • cc deleted.
  • status changed from new to assigned.

Also, please post the /tmp/realtime-changes.txt so I can quickly see whether it's the file itself which is bad (would prove whether it's capture output vs integrity.pm parsing)

In the mean time, I will add sanity checks on the integrity.pm side

11/06/07 10:45:38 changed by kindlund

This issue was reproducible while the wimprvse.exe did maintenance work; I'm not sure how easily it will be able to reproduce today, but we can try.

This is the exact output of the realtime-changes.txt file:

"2007-11-05 18:08:43.875","process","created","976","C:\WINDOWS\system32\svchost.exe","332","C:\WINDOWS\system32\wbem\wm
iadap.exe"
"2007-11-05 18:08:51.15","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REG","WMIBinaryMofR
esource.HighDateTime=29653422,LowDateTime=2875513088,Name="C:\\WINDOWS\\System32\\DRIVERS\\ACPI.sys[ACPIMOFResource]"","
REG_NONE",""
"2007-11-05 18:08:51.219","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REG","WMIBinaryMof
Resource.HighDateTime=29653447,LowDateTime=3591330688,Name="C:\\WINDOWS\\System32\\DRIVERS\\mssmbios.sys[MofResource]"",
"REG_NONE",""
"2007-11-05 18:08:51.703","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REGISTRY\","WMIBin
aryMofResource.HighDateTime=29435636,LowDateTime=863039744,Name="C:\\WINDOWS\\System32\\DRIVERS\\pcntpci5.sys[NdisMofRes
ource]"","REG_NONE",""
"2007-11-05 18:08:51.828","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","\REGISTR","WMIBinar
yMofResource.HighDateTime=29653422,LowDateTime=1215513088,Name="C:\\WINDOWS\\System32\\DRIVERS\\ipnat.sys[IPNATMofResour
ce]"","REG_NONE",""
"2007-11-05 18:08:51.953","registry","DeleteValueKey","780","C:\WINDOWS\system32\wbem\wmiprvse.exe","","WMIBinaryMofReso
urce.HighDateTime=29666671,LowDateTime=3283808384,Name="C:\\WINDOWS\\System32\\Drivers\\HTTP.sys[UlMofResource]"","REG_N
ONE",""

— Darien

11/06/07 14:02:41 changed by kindlund

  • attachment realtime-changes.txt added.

Realtime Changes Output

11/08/07 13:02:39 changed by xkovah

  • status changed from assigned to closed.
  • resolution set to fixed.

Fixed via fixing capture code + adding sanitization code to make sure there can be no null key_name incase something else goes wrong somewhere.


Add/Change #117 (DB Insertion Error - Possible Corrupt Capture Parsing?)




Change Properties
Action