Showing posts with label behavior. Show all posts
Showing posts with label behavior. Show all posts
Friday, September 29, 2017
User behavior related work
User behavior related work
Many enterprises already recognized the importance of protecting against various threats that anti viruses cant predict but also the high probability of inside-out threats that pass undetected. Event if a threat is detected, most organizations arent equipped to respond effectively. Because of that, different user behavior based security software has been built and usually named under a general term User and entity behavior analytics (UEBA) software. It can be defined as: "Analysis of the behaviors of organizations employees, outsiders connected to their networks (such as third party contractors) and flagging security vulnerabilities across organizations assets that hold sensitive data.".
The difference between UEBA and the rest of the similar security software is that it usually can quickly identify a threat or an exploitable asset and then take action to remediate security risks across the entire infrastructure. Here Ill list a few examples of some software for different purposes that offer such features.
SentinelOne
Software that can be deployed across Windows, OS X and Linux endpoints along with Linux Ubuntu management server. They offer:
- monitoring kernel and user space (files, processes, system calls, memory, registry, network etc.). More details can be found in their technical brief.
- rapidly eliminating threat by killing malicious processes, rolling back manipulated files, disconnecting compromised devices,
- real-time attack forensic analysis.
Platform that seems to be supporting Windows OS only. They offer:
- monitoring following data sources: VPN, FW, IPS/IDS, web proxy, email logs,� packets, DNS logs, Active Directory logs, �DHCP logs,
- detecting privilege escalation, credential violations, internal reconnaissance, lateral movement, abnormal access to high value resources, command and control, exfiltration,
- alerts classified by severity and attack stage.
Platform that supports Windows & NAS, Exchange, Active Directory, SharePoint, UNIX/Linux, Office 365. Mostly offers detecting security gaps and insider threats by tracking changes to important configuration files, access to sensitive files, malware, privilege escalations, access denied events and more. Also includes:
- monitoring files and emails,
- full visibility on permissions (folder, mailbox, sharepoint),
- real-time alerts and comprehensive auditing.
Platform contains their own implementation of syslog-ng log management solution but doesnt offer much information about the data sources. Their technical documentation can be found here, and the features they offer include:
- analyzing biometric information (typing style or typical mouse movements),
- automatic notifications based on top suspicious activities.
This solution helps in protecting online banking sites against account takeover, fraudulent transactions, and can detect end user devices infected with high risk malware. It includes:
- analyzing biometric information (subtle mouse movements and clicks),
- possible integration with mobile devices for analysis of malware infections, root and jailbroken information, accurate geolocation and Wi-Fi security status.
- protection of web browser sessions to prevent tampering of customer transactions,
- prevention of phishing attacks, malware infections and removal of existing malware,
- protection against phishing of login credentials and payment card data.
This threat analytics platform offers insight into endpoints, applications, devices and users. Its benefits are:
- identifying and predicting malicious insiders and comprised accounts
- detecting and blocking fraud by proactively alerting on anomalous behaviors,
- real-time contextual view of attacks and detailed reports
download file now
Friday, September 22, 2017
User behavior logging
User behavior logging
In this blog post some ways to examine Linux operating system without using any additional (third party) programs or tools are explained. There are a number of Linux distributions but I will focus primarily on Debian and Debian based distributions. By collecting all the available information it is possible to track and log user behavior. Post consists of two parts: default system logs and other aspects which could be monitored.
In this part of the post some of the significant existing Linux system logs for user behavior are listed and explained:
- /var/log/messsages - contains general system activity and non-critical messages like user logins, kernel messages, IP firewall packet logging and so on. This log file doesnt exist on Debian Linux distributions, so instead syslog is used.
- /var/log/syslog - contains all the messages except the authentication related ones. By analyzing few, I found only kernel and thermald messages. Each line contains: datetime, hostname, program that generated the message, process id and log message.
- /var/log/auth.log - contains system authorization information including user logins through display and login managers, sudo access requests, authentication mechanism for crontab, policykit system daemon etc. This log file is found on Debian Linux distributions, but some other use /var/log/secure instead.
- /var/log/btmp - keeps track of failed login attempts. It is a binary file and can be read using last command.
- /var/log/dpkg.log & /var/log/yum.log - contain messages about installs or upgrades for various package managers.
It is important to point out that not all Linux distributions have those logs enabled. /etc/rsyslog.conf file controls what goes inside the log files while /etc/rsyslog.d/*.conf configuration files control what goes in log directory and where that directory is (default value is /var/log).
Also, log files can be easily modified or deleted. One solution is to set the log files to "append only" with chattr +a, but an attacker can gain root access so it is better to send logs to another host.
Other ways for tracking user behavior
Integrity checking is another way to detect changes in the system by looking for changes in the MD5/SHA1 checksums of the key files in the system. Those checksums need to be calculated periodically and compared with previous values. In such way inotify tool works by monitoring individual files or directories. It is important to check metadata such as permissions and ownerships, not just changes to file contents.
Besides monitoring logs for package installs, browser extensions are almost equally good for tracking user behavior. This can be done by checking integrity of directory where browser stores plugins, for example on Google Chrome it is ~/.config/google-chrome/Default/Extensions/ but on Mozilla Firefox it is /usr/lib/firefox-addons.
Running programs are easy to track and analyze because Linux contains virtual filesystem /proc with all process information. Directories inside /proc correspond to an actual process ID and by using command such as ps ax we can match processes with the associated process ID. Each of these directories contain information about command line arguments, CPU, current working directory, environment variables, links to the executables and library files of the process, memory held and process status. Directory /proc contains a lot more system information that can be analyzed such as: filesystems, IDE devices, memory map, I/O ports, general network information, parallel ports, partitions, general kernel behaviors, network core and interrupts.
Keyboard is the most used device by the user so that should be logged too (keypress and speed). There are also shell commands which are sensitive part of all Unix systems. They can be monitored in several ways but one of them is to use the command history. However, there are way too many ways to execute a command in any Unix system so it is not possible to monitor commands completely.
Since mostly users browse same websites daily, it can be useful for determining users behavior to log website visits. That can be done using rndc querylog which can log all DNS queries or using tcpdump and filtering only DNS queries (port 53).
Those are some of the ways to examine the status of some system and by that we can try to track user behavior. Also, there are some other parts of the system that are not mentioned but that is because I currently dont find them significant for purpose of the system I am building.
Keyboard is the most used device by the user so that should be logged too (keypress and speed). There are also shell commands which are sensitive part of all Unix systems. They can be monitored in several ways but one of them is to use the command history. However, there are way too many ways to execute a command in any Unix system so it is not possible to monitor commands completely.
Since mostly users browse same websites daily, it can be useful for determining users behavior to log website visits. That can be done using rndc querylog which can log all DNS queries or using tcpdump and filtering only DNS queries (port 53).
Those are some of the ways to examine the status of some system and by that we can try to track user behavior. Also, there are some other parts of the system that are not mentioned but that is because I currently dont find them significant for purpose of the system I am building.
download file now
Friday, September 15, 2017
User behavior data extraction in Linux OS
User behavior data extraction in Linux OS
In this post, a short intro about my seminar in masters degree programme is given. The topic that Im working on is "User behavior data extraction in Linux OS" which includes gathering useful data about user actions, parsing and analyzing them.
The purpose of the system is to gather all information such as global system messages, user logins, background daemons, package installs, browser and keyboard usage, shell commands, runtime system information, file integrity and so on. Then, by using machine learning, it might be possible to develop a system that can detect anomalous user behavior. Such system would try to prevent any unauthorized access by analyzing users current activities.
The following activities and their approximate duration are planned:
- search for similar software (5%)
- Linux logging analysis (10%)
- finding all options to track users behavior (15%)
- definition and design of desired system (15%)
- implementation in Python programming language (40%)
- testing systems functionality and security (15%)
download file now
Tuesday, September 5, 2017
Using Google Analytics to understand real time messaging behavior
Using Google Analytics to understand real time messaging behavior
This is a guest post by Nico Miceli, a Google Developer Expert for Google Analytics, Technical Analytics Consultant on Team Demystified, quantified selfer, and all around curious guy. He blogs at nicomiceli.com and tweets from @nicomiceli.
Hello, my name is Nico, and I love data. I quantify everything, and the Google Analytics Measurement Protocol is my favorite way to do it.
With the Measurement Protocol, I can send, store, and visualize any data I want without having to build a backend collection system. I�ve even used it in my personal life to track my sleep patterns, the temperature in my house, and the number of times my brother�s cat actually uses his scratching post.
So when my team started using Slack, a real-time messaging app for teams, I wanted to get the stats. Which clients are contacting us most frequently? When are the contacting us? More importantly, who on our team is the wordiest and uses the most emojis? Out of the box, the app offered some data, but it wasn�t enough for me to answer all the questions I had.
After taking a look at the technical documentation for the messaging app, I realized that Google Analytics is the answer! With the Measurement Protocol and the Slack Real Time API, I could get SO MUCH DATA!! With help from fellow developer Joe Zeoli, Slackalytics was born.
Because the data gets stored in Google Analytics, you can visualized and analyze within the UI or use the Google Analytics Core Reporting API. I like to combine this data with other information so I have export it all into a Google sheet using the Google Analytics Spreadsheets Add-on.
In this beta version of Slackalytics, I�m using two Custom Dimensions: User ID, Channel Name... and six Custom Metrics: Word Count, Letter Count, Emoji Count :), Exclamation Count !!!, Question Count ???, Ellipse Count...
But this is just a fraction of what�s possible. Slackalytics is open source, so you can build your own version. If you�re a developer: Fork my project on GitHub.
If you�re not a developer: Fear not. You can still create your own messaging analysis bot by following my detailed walkthrough on setting this up.
Developer or not, you can build and test your own bot by using Google Analytics and any communication app that has a realtime API. Find out when your clients ask the most questions, monitor other integrations and bots, find out who talks in
or build your own new Custom Dimension & Metrics combos.
- The Google Analytics Developer Relations team, on behalf of Nico Miceli
Hello, my name is Nico, and I love data. I quantify everything, and the Google Analytics Measurement Protocol is my favorite way to do it.
With the Measurement Protocol, I can send, store, and visualize any data I want without having to build a backend collection system. I�ve even used it in my personal life to track my sleep patterns, the temperature in my house, and the number of times my brother�s cat actually uses his scratching post.
So when my team started using Slack, a real-time messaging app for teams, I wanted to get the stats. Which clients are contacting us most frequently? When are the contacting us? More importantly, who on our team is the wordiest and uses the most emojis? Out of the box, the app offered some data, but it wasn�t enough for me to answer all the questions I had.
After taking a look at the technical documentation for the messaging app, I realized that Google Analytics is the answer! With the Measurement Protocol and the Slack Real Time API, I could get SO MUCH DATA!! With help from fellow developer Joe Zeoli, Slackalytics was born.
Slackalytics (in beta) is a simple, open source bot for analyzing Slack messages. Built in node.js, it grabs messages from Slack (using the Slack Real Time Messaging API), does some textual analysis, and counts the occurrences of specific instances of words and symbols. Then, using the Measurement Protocol, it sends the data to your Google Analytics account.
Screenshot of the report showing the custom metrics (emoji, exclamation, word, and ellipse counts) for different Slack channels.
Because the data gets stored in Google Analytics, you can visualized and analyze within the UI or use the Google Analytics Core Reporting API. I like to combine this data with other information so I have export it all into a Google sheet using the Google Analytics Spreadsheets Add-on.
In this beta version of Slackalytics, I�m using two Custom Dimensions: User ID, Channel Name... and six Custom Metrics: Word Count, Letter Count, Emoji Count :), Exclamation Count !!!, Question Count ???, Ellipse Count...
But this is just a fraction of what�s possible. Slackalytics is open source, so you can build your own version. If you�re a developer: Fork my project on GitHub.
If you�re not a developer: Fear not. You can still create your own messaging analysis bot by following my detailed walkthrough on setting this up.
Developer or not, you can build and test your own bot by using Google Analytics and any communication app that has a realtime API. Find out when your clients ask the most questions, monitor other integrations and bots, find out who talks in
or build your own new Custom Dimension & Metrics combos.- The Google Analytics Developer Relations team, on behalf of Nico Miceli
download file now
Subscribe to:
Posts (Atom)