月度归档:2018年01月

debian/ubuntu install 安装 ffmpeg

ffmpeg太出名了,linux上只要跟多媒体有关的多多少少都会有它的身影。最近需要用到它读取多媒体metadata,使用的库说找不到ffmpeg,apt也装不了会报错误。原来debian7开始ffmpeg被libav取代,这个东西也是从ffmpeg fork出来的,到了debian9又回到了ffmpeg,不过奇怪的是我换了stretch还是无法安装ffmpeg。老版本如果要装ffmpeg可以加入backports源,例如jessie源可以加入如下:

 
deb http://mirrors.163.com/debian/ jessie-backports main non-free contrib
deb-src http://mirrors.163.com/debian/ jessie-backports main non-free contrib

 

 

阿里云OSS js SDK使用https出现ERR_SSL_PROTOCOL_ERROR错误


在HTTPS站点ajax请求的地址如果不是https那么就有ERR_SSL_PROTOCOL_ERROR错误了。

解决办法:当然是把http改为https即可

阿里云一贯loss文档的尿性,确实找不到相应的说明,甚至找到对应sdk的项目地址都难,这里贴上oss js sdk的地址,看去是淘宝ued出品:https://github.com/aliyun-UED/aliyun-sdk-js

找到后发现有n多可配置项,这里需要加secure:true 到oss对象初始化的options即可,默认是false所以请求的是http。

最后重要的事说三遍:鄙视👎阿里云、鄙视👎阿里云、鄙视👎阿里云

跨flask app使用url_for

用flask的肯定知道url_for,解放了记住url的繁琐。前阵子因为flask臭名昭著的 servername问题彻底拆分了应用程序,在解决了大部分引用和重用后发现应用间导航也是个问题,写硬url显然不科学。这里祭出几大法宝用来解决这个问题:

  1. 中间件Dispatcher,也可以使用werkzeug.wsgi 的 DispatcherMiddleware,中间件的设计理念真是扩展所必须的呀;
  2. with nebapp.app_context,没错在每个应用程序中创建邻居app后可以使用它的上下文即可用url_for了;
  3. 实际使用会有报错,这里需要mock一个request,这里flask准备好了app.test_request_context;

贴一下我在用的Dispatcher

class Dispatcher:
    """
    Allows one to mount middlewares or applications in a WSGI application.

    This is useful if you want to combine multiple WSGI applications::

        app = DispatcherMiddleware(app, {
            '/app2':        app2,
            '/app3':        app3
        })

    """

    def __init__(self, app, mounts=None):
        self.app = app
        self.mounts = mounts or {}
        self.url_for_resolver = URLForResolver(
            self.mounts
        )
        self.app.dispatcher = self

    def __call__(self, environ, start_response):
        script = environ.get('PATH_INFO', '')
        path_info = ''

        while '/' in script:
            if script in self.mounts:
                app = self.mounts[script]
                break
            script, last_item = script.rsplit('/', 1)
            path_info = '/%s%s' % (last_item, path_info)
        else:
            app = self.mounts.get(script, self.app)

        original_script_name = environ.get('SCRIPT_NAME', '')
        environ['SCRIPT_NAME'] = original_script_name + script

        # Convert empty path info values to a forward slash '/'
        environ['PATH_INFO'] = path_info or '/'

        return app(environ, start_response)


class URLForResolver:
    """
    A URL resolver that provides resolution of `url_for` across multiple apps.
    """

    def __init__(self, mounts):
        self.mounts=mounts
        self.apps = list(mounts.values())
        self.cache = {}

        for app in self.apps:
            app.url_build_error_handlers.append(self)

    def __call__(self, endpoint, **values):
        """Attempt to resolve a URL any of the registered apps"""
        error = Exception('can not find endpoint for %s' % endpoint)

        parts=endpoint.split('.')
        path=parts.pop()
        if len(parts)>1:
            path='%s.%s'%(parts[1],path)
        app_name=None
        if len(parts)>0:
            app_name=parts[0]

        # Check if we have a cached look up
        if endpoint in self.cache:
            app = self.cache[endpoint]
            if app:
                with app.app_context(), app.test_request_context():
                    ret_path= url_for(path, **values)
                    if ':5000' in ret_path:
                        ret_path=ret_path.replace(':5000','')
                    return ret_path
            else:
                raise error

        if app_name:
            app = self.mounts[app_name]
            self.cache[endpoint] = app
            return self(endpoint, **values)
        # Attempt to find an app with the registered endpoint
        for app in self.apps:

            for rule in app.url_map.iter_rules():

                if rule.endpoint == path:
                    # Found - cache the result and call self to return the URL
                    self.cache[endpoint] = app
                    return self(endpoint, **values)

        # Not found - cache the result and re-raise the error
        self.cache[endpoint] = None
        raise error

使用方式:

#初始化
init_apps()
    apps = create_apps()
    del apps['/passport']
    dispatcher = Dispatcher(passport, mounts=apps)
#调用
current_app.dispatcher.url_for_resolver('/appname.blueprint_name.page_name',_external=True))

pycharm remote docker debug with external sources

 

Great IDE pycharm can easy debug with docker-compose.yml file. But default guid not metioned how to config debug with external sources. If Problem happend in external sources ,I have to stupid add sources to my project for debug. After Google a lot ,it seems like path mapping problem. What important is the breakpoint you locate can not be in pycharm cache, you can check this when move cursor to the source tab shows the path.

Solution(Menu:Pycharm>Project Interpreter):

  1. Click dots in the end of “Path mappings”
  2. find your local python path ,if you using py3 in mac the path should looks like “/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4”
  3. Add two mappings ,below is my config

“/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-package” : “/usr/local/lib/python3.4/site-package”

“/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4″:”/usr/local/lib/python3.4”

After all done you need to referesh remote sources, maybe not but i did. Then pycharm will reindex soruces ,wait a few minites, you can check with command+B jump to external sources , if path changed to your local sites-package location its success.

用户密码不要再用MD5加密了,不妨改为Bcrypt

 

MD5实在太出名了,特别是刚毕业的同学脑子里就是Hash,MD5,AES几种,其实算法的世界就跟森林一样,还每一颗都不同。各有各的特长和适用场景,多年前MD5因为计算的复杂度一度被认为不可破解,但是随着cpu的摩尔定律现在已经可以被破解了。玩SS的同学不妨可以切换几种试试,小众的可能被侦测的概率会更小。

回归正题,黑客拿到网站数据库后当然就是破解用户的加密密码,这个时候MD5加密的密码网站上有专门的rainbow服务提供碰撞破解,甚至常见的人家都已经cache了,碰撞都不需要。道高一尺魔高一丈想不被破解除了采用加salt这类亡羊补牢的办法外更换算法才是最有效果的。DANG!DANG!有请主角Bcrypt出场,前面MD5之所以被破解是因为它的复杂计算后来可以被CUDA方式加速计算,结果就是可能1ms就够了,对于高级显卡个人PCrainbow都没问题。

所以bcrypt的特点之一就是不能被gpu加速;特点二本身要够慢,当破解需要几十上百年的时候也就可以认为它不可被破解,所以算法本身除了复杂度外怎么就是更慢,对于算法来说要做到这个可不容易,bcrypt采用了Blowfish的算法,让他慢的在算法里都显得特别奇葩,前面1ms的加密它可能需要0.3秒。算法终究是会过时的,bcrypt为了延长生命期还增加了salt和work factor,后者可以自由更改用于增加因为cpu性能增加让系统计算更慢。

‘$2b$12$nf3TdEFJZ6AtRA7N9r1f.OANfpFS9Qxtklo4km.LHbuEV7Mu5jJN2’

上面就是一段用bcrypt加密‘mypassword’后的结果,这里$是分隔符没有意义,2b代表算法版本,12代表work factor。

对于一些大型站点可能不会用bcrypt毕竟它太慢,但是对于中小型站点来说它足够安全且用于频率很低的密码还是很不错的选择。

PS:wp和gitlab都是用的bcrypt。

绕过目标网站对Ad Block 的检测

我想用ab的同学一定碰到过提示加入ab的白名单的标语,甚至有的会阻碍你的浏览如果不加入白名单。开始的时候对有的网站本身广告较少,排版合理的我都会乐意加入白名单,但是当这样的提示越来越多,并且严重影响阅读的广告越来越多后就痛苦了,毕竟经常上的网站就那么几个。这个时候神奇油猴就派上用场了,更开心的是排名超前的aak很完美的实现了这个功能,地址:https://openuserjs.org/scripts/reek/Anti-Adblock_Killer_Reek

可以继续无广告玩耍了,当然对于喜欢的网站还是会妥妥的加入白名单的。