technology.txt

https://www.iana.org/assignments/media-types/media-types.xhtml

xcopy "$(ProjectDir)www" "$(SolutionDir)RuiJi.Net.Cmd\bin\debug\www" /E /Y


open source projects that may be possible to use in the future

cassandra
Cassandra is a high performance distributed key value store.
Install-Package Cassandra -Version 2.2.5	

Phantomjs
PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.
Install-Package PhantomJS -Version 2.1.1

casperjs

Hadoop
Spark 

数据采集
Sqoop
Flume
HUB web管理
Oozie 工作流调度

数据存储 ✓X
函数添加 ✓
Selector结果处理函数 ✓
日志 ✓X
代理服务器管理 ✓
Web Ui ✓X
Seed需要分两种类型：检测更新型/即时抓取型 ✓X
Cookie信道 ✓
Cookie API X
自动化登陆 X (https://slimerjs.org/  or  https://github.com/ariya/phantomjs/blob/master/examples/phantomwebintro.js )
动态类生成 X
分离config文件 X

Redis X
Orc(Tesseract orc) X
RuiJi Expresss advanced setting X
https proxy X
Java/PHP/C# SDK X
Download result to be packaged into zip X

--------------------------------------------

Storage
Spider
Cookie Chanel ✓
Post Ajax ✓
Logger
RuiJi Expresss advanced parser
https proxy
File Func ✓
RTS X
MessageQueue iisue

-------------------------------------------
Zk cluster/HA

任务调度
规则缓存