DAMICORE files uploaded.

Author: nilsonsangy

.gitignore

@@ -1,4 +1,9 @@
 .ipynb_checkpoints/
 .vagrant/
+.vscode
+__pycache__
 hydra.restore
 host-nastware.log
+damicore/bkp
+damicore/erro.txt
+damicore/tmp/

damicore/LICENSE

@@ -0,0 +1,84 @@
+# DAMICORE
+
+  DAMICORE is an easy-to-use clustering and classification tool.
+
+  DAMICORE is a pipeline of algorithms which is agnostic to the type of data
+thanks to NCD, a compressor-based metric which views any piece of data as
+simply a string of bits. This is particularly well-suited for heterogenous
+datasets, datasets with difficult characteristics extraction and text datasets.
+
+     Data    Metric      Distance matrix    Simplification Phylogenetic tree
+    -----.           .---------------------.              .-----------------.
+     x_1 |           |     x_1 x_2 ... x_n |              |   x_1-.  x_n    |
+     x_2 |  .-----.  | x_1 d11 d12     d1n |    .----.    |        \  |     |
+     x_3 |->| NCD |->| x_2 d21 d22     d2n |--->| NJ |--->|         O-O---- |-->
+     ... |  '-----'  | ...                 |    '----'    |        /        |
+     x_n |           | x_n dn1 dn2     dnn |              | x_2---'         |
+    -----'           '---------------------'              '-----------------'
+   
+            Community detection        Clusters
+    -----.                        .-----------------.
+         |                        |  ___       ___  |
+         |    .-------------.     | |x_1|     |   | |
+    tree |--->| Fast Newman |---->| |x_2| ... |C_m| |
+         |    '-------------'     | |x_n|     |   | |
+         |                        | '---'     '---' |
+    -----'                        '-----------------'
+
+
+## Dependencies
+
+### Installation
+* python 3.10
+* igraph library (visit http://igraph.org/c for more informations)
+* python-igraph (visit http://igraph.org/python for more informations)
+
+### Execution
+
+  DAMICORE relies on compressors to calculate the distance between a pair of
+objects (files).
+
+* gzip (available in almost all \*nix systems)
+* bzip2 (available in almost all \*nix systems)
+
+### Recommended
+
+* FigTree (available in http://tree.bio.ed.ac.uk/software/figtree/)
+
+## Usage
+
+The simplest way to use DAMICORE is to provide as argument a directory
+containing all files to be compared:
+
+     $ ./damicore.py examples/texts
+
+This outputs a list of pairs of file name and corresponding cluster index.
+For now we are lacking a good tool to visualize this clustering, but there are
+other tools that might help. We can output intermediate steps into different
+files for analysis:
+
+     $ ./damicore.py examples/texts --ncd-output texts.phylip --format phylip \
+     --tree-output texts.newick --graph-image texts.png --output texts.clusters \
+     --compress gzip
+
+This outputs the NCD matrix using PHYLIP format, the neighbor-joining tree in
+Newick format (readable by FigTree), an image with a graph visualization and
+the final clusters into another file.
+
+By default, the script uses PPMd as compressor. If you don't have it installed,
+try using gzip or bzip2:
+
+     $ ./damicore.py examples/texts --compressor gzip
+
+For more information on available options, see --help.
+
+## Contact
+
+If you believe you have found a bug, or would like to ask for a feature, please
+inform me at [email protected].
+
+## License
+
+This software is licensed under the GPLv2.
+Texts examples dataset collected by Francisco José Monaco ([email protected])
+More informations in https://github.com/brunokim/damicore-python .

damicore/README.md

@@ -0,0 +1,142 @@
+US begins government shutdown as budget deadline passes
+Previous
+Latest
+Who's affected
+Analysis
+Storm clouds
+60-sec explainer
+Q&A
+Economy
+Next
+COMMENTS (219)
+
+What does shutdown mean for two million federal employees, agencies and tourist destinations?
+Continue reading the main story
+US budget row
+
+Who is affected
+Q&A
+Mardell analysis
+Impact
+The US government has begun a partial shutdown after the two houses of Congress failed to agree a new budget.
+
+The Republican-led House of Representatives insisted on delaying President Barack Obama's healthcare reform - dubbed Obamacare - as a condition for passing a bill.
+
+More than 700,000 federal employees face unpaid leave with no guarantee of back pay once the deadlock is over.
+
+It is the first shutdown in 17 years and the dollar fell early on Tuesday.
+
+Continue reading the main story
+“
+Start Quote
+
+The Republican leadership looks and feels trapped - they made demands that they knew wouldn't be met rather than be accused of weakness and betrayal by their own hardliners”
+
+image of Mark Mardell
+Mark Mardell
+North America editor
+Read more from Mark
+Goldman Sachs estimates a three-week shutdown could shave as much as 0.9% from US GDP this quarter.
+
+On Tuesday, Mr Obama blamed the House of Representatives for the stalemate and said he would "keep working to get Congress to reopen the government [and] restart vital services".
+
+"This shutdown was completely preventable. It should not have happened," he wrote in a letter to federal government employees.
+
+"And the House of Representatives can end it as soon as it follows the Senate's lead, and funds your work in the United States Government without trying to attach highly controversial and partisan measures in the process."
+
+On Monday, House Speaker John Boehner told reporters he hoped the Senate would agree to a committee between the two chambers known as a conference "so we can resolve this for the American people".
+
+"The House has voted to keep the government open but we also want basic fairness for all Americans under Obamacare," he said.
+
+But on Tuesday morning, the Senate voted 54-46 to reject the request for formal negotiations to end the impasse.
+
+Who is affected?
+Figures
+Continue reading the main story
+State department will be able to operate for limited time
+Department of defence will continue military operations
+Department of education will still distribute $22bn (£13.6bn) to public schools, but staffing is expected to be severely hit
+Department of energy - 12,700 staff expected to be sent home, with 1,113 remaining to oversee nuclear arsenal
+Department of health and human services expected to send home more than half of staff
+The Federal Reserve, dept of homeland security, and justice dept will see little or no disruption
+US Postal Services continue as normal
+Smithsonian institutions, museums, zoos and many national parks will close
+US shutdown in 60 seconds
+Who will be affected
+Q&A: 2013 US budget brawl
+Continue reading the main story
+Timeline: US budget crisis
+
+20 September: House votes to scrap funding for Affordable Care Act ("Obamacare")
+30 September: Congress passes two budget bills coupled to Obamacare, both rejected by Senate
+1 October: Key provisions of Obamacare come into force despite shutdown
+17 October: Deadline for extending government borrowing limit, or debt ceiling
+The BBC's Mark Mardell in Washington says the divide in US politics has grown so bitter that government itself cannot function.
+
+Democrats were never likely to make concessions on healthcare reform - Mr Obama's signature achievement and a central issue in last year's presidential election, our correspondent says.
+
+But Republicans have made demands that they knew would not be met rather than be accused of weakness and betrayal by their own hardliners, he adds.
+
+
+Engineers with the US Navy talk to the BBC about what they will do during a shutdown: Make skis
+On Monday, the Democratic-led Senate twice rejected bills from House Republicans that would have funded the government only if funding for President Obama's healthcare law was delayed for a year.
+
+Major portions of the healthcare law, which passed in 2010 and has been validated by the US Supreme Court, took effect on Tuesday regardless of whether there is a shutdown.
+
+President Obama went on national television to criticise Republicans for trying to refight the last election.
+
+
+Obama accused Republicans of demanding ransom over Obamacare
+A shutdown would have "a very real economic impact on real people, right away," he said, adding it would "throw a wrench" into the US recovery.
+
+"The idea of putting the American people's hard-earned progress at risk is the height of irresponsibility, and it doesn't have to happen."
+
+As the shutdown neared, the Senate's Democratic majority leader blamed Republicans for the imminent halt to all non-essential government operations.
+
+"It will be a Republican government shutdown, pure and simple," said Harry Reid, referring to the Republicans as "bullies".
+
+A US Park Police officer watches at left as a National Park Service employee posts a sign on a barricade closing access to the Lincoln Memorial in Washington on 1 October 2013.
+The White House issued orders for government offices to start shutting down as the midnight deadline approached
+Tourists look at a sign announcing that the Statue of Liberty is closed due to a US government shutdown in New York, on 1 October 2013.
+Tourism revenues are expected to take a hit as attractions such as the Statue of Liberty close down
+Signpost announcing closure of Library of Congress in Washington DC on 1 October
+National parks, museums, federal buildings and government services have been shut down indefinitely
+A furloughed federal employee holds a sign on the steps to the US Capitol after the US government shut down last night, on Capitol Hill in Washington on 1 October 2013.
+Some 700,000 federal workers have been sent home on unpaid leave
+A barricade leading to the Lincoln Memorial prevents access to tourist buses in Washington on 1 October 2013.
+Some White House staff have been sent home, with all Smithsonian institutions in Washington closed
+Mr Obama has signed legislation ensuring that military personnel would be paid. The defence department had advised employees that uniformed members of the military would continue on normal duty, but that large numbers of civilian workers would be told to stay home.
+
+Continue reading the main story
+US media
+
+Michael Gerson, Washington Post: We are no longer seeing a revolt against the Republican leadership, or even against the Republican "establishment"; this revolt is against anyone who accepts the constraints of political reality.
+
+Editorial, Chicago Tribune: Once Republicans and Democrats stop grandstanding... we think there's an easy solution to this impasse.
+
+Continue reading the main story
+International media
+
+Pierre-Yves Dugua, Le Figaro, France: The [US] is being humiliated by the inability of its political system to carry out its primary mission: to pass a budget.
+
+Patrick Welter, Frankfurter Allgemeine Zeitung, Germany: The failure to reach agreement casts a dark shadow on the next and more important forthcoming round in the fiscal row.
+
+Andrew Coyne, National Post, Canada: The result of all this haggling, taken together, has been just the sort of balanced approach, mixing spending cuts and tax increases, that experts advise.
+
+Under the shutdown, national parks and Washington's Smithsonian museums will close, pension and veterans' benefit cheques will be delayed, and visa and passport applications will go unprocessed.
+
+Programmes deemed essential, such as air traffic control and food inspections, will continue.
+
+The US government has not undergone a shutdown since 1995-96, when services were suspended for a record 21 days.
+
+Republicans demanded then-President Bill Clinton agree to their version of a balanced budget.
+
+As lawmakers grappled with the latest shutdown, the 17 October deadline for extending the government's borrowing limit looms even larger.
+
+On that date, the US government will reach the limit at which it can borrow money to pay its bills, the so-called debt ceiling.
+
+House Republicans have also demanded a series of policy concessions - including on the president's health law and on financial and environmental regulations - in exchange for raising the debt ceiling.
+
+Guy Crundwell from Connecticut told the BBC that politicians should be solving the country's problems rather than engaging in a "charade".
+
+"I am very fiscally conservative but for moral issues I lean towards the Democrats, but I'll be damned if I want to see either of them wasting my money on this sort of posturing."

damicore/examples/texts/ENG-TXT-1.txt

@@ -0,0 +1,51 @@
+Pupils breakfasting on energy drinks
+By Hannah Richardson
+BBC News education reporter
+Energy drinks
+Energy drinks can have very high levels of caffeine and are not usually recommended for children
+Continue reading the main story
+Related Stories
+
+Parents 'fail to give breakfast'
+Call for free school breakfasts
+One in 20 teenage pupils goes to school on a can of energy drink instead of a good breakfast, a survey suggests.
+
+A third of UK teenagers eat unhealthy breakfasts such as fry-ups or crisps, the poll of 2,000 youngsters aged between 12 and 18 suggests.
+
+Four out of 10 skip breakfast altogether to lose weight, the poll for the Make Mine Milk campaign suggests.
+
+Brian Lightman, general secretary of the Association of School and College Leaders, said the results were a worry.
+
+He said pupils breakfasting on energy drinks, which are not usually recommended for children, was something he had seen as a head teacher.
+
+'Hyperactive'
+"When children arrive in schools having started the day eating or drinking totally inappropriate things like that, they are in no fit state to be in the classroom.
+
+"They can be hyperactive, and it can have a very negative effect on their behaviour."
+
+He added that parents should be ensuring that their children were eating good breakfasts, rather than sending them off with money to buy something on the way to school.
+
+He added: "There's a lot of advertising with these drinks and they are fashionable, and teenagers love things that are fashionable."
+
+Dr Mary Bousted, general secretary of the Association of Teachers and Lecturers (ATL), also said it was vital that children had a good and nutritious breakfast to set them up for the day and enable them to concentrate in class.
+
+"A healthy breakfast is also important in stopping children become fractious and badly behaved.
+
+"It is worrying to think that some children are breakfasting on energy drinks, such as Red Bull, which are stuffed full of sugar and stimulants and have little nutritional value.
+
+"School breakfast clubs can play a key role in making sure children have had a good breakfast before they start school to help their learning. And we would like children to be taught about healthy eating, so they learn about healthy diets and how to cook nutritious meals."
+
+Caffeine content in such energy drinks ranges from about 8mg of caffeine - about the same as an espresso coffee - to as much as 400mg.
+
+'Shot style'
+Some of the smaller "shot style" products can contain as much as 175mg of caffeine in a 60ml bottle.
+
+Drinks containing more than 150mg of caffeine per litre (mg/l) must be labelled with the term "high caffeine content", and the Food Standards Agency recommends that children should only "consume in moderation drinks with high levels of caffeine".
+
+The makers of the Red Bull Energy Drink did not wish to comment on the issue raised but its website says: "Red Bull Energy Drink's special formula has been appreciated worldwide by top athletes, students, and when undertaking demanding work or during long drives."
+
+It adds the drink is "the ideal equipment when you are active and keen to set personal bests".
+
+Director general of the British Soft Drinks Association Gavin Partington said: "The soft drinks industry is clear that energy drinks are not suitable for children, and we want to get that message across to young people and their parents.
+
+"Adults might choose an energy drink, containing about as much caffeine as a typical cup of coffee, when they want a physical or mental boost. But, like all food and drink, energy drinks should be consumed in moderation as part of a balanced diet and active lifestyle."

damicore/examples/texts/ENG-TXT-10.txt

@@ -0,0 +1,51 @@
+UK in Afghanistan transformed country, says commander
+british soldiers in afghanistan
+All British combat operations are due to finish by the end of 2014
+Continue reading the main story
+UK troops in Afghanistan
+
+Military deaths in Afghanistan
+UK packs up equipment in Afghanistan
+Roadside bombs 'deadliest weapon'
+Changing role of UK troops
+British sacrifices have transformed Afghanistan, an Army commander says.
+
+Brig Rupert Jones said the drawdown of UK troops was progressing well and that the British people should be proud of what they have achieved in the country.
+
+The UK had given the Afghans "an opportunity", he said, but warned it was "down to them" how they used it.
+
+Brig Jones is commander of Task Force Helmand, whose HQ moved from Lashkar Gah to Camp Bastion in summer. He is due to hand over to Brig James Woodham.
+
+'Lot of effort'
+Almost 8,000 British troops are still serving in Afghanistan, around half of them based at the camp.
+
+The number is scheduled to fall to 5,200 by the end of the year.
+
+All UK combat operations are due to finish by the end of 2014, with responsibility transferred to Afghan forces.
+
+Since the war began 12 years ago, 444 UK service personnel have lost their lives in the fight against the Taliban.
+
+"I think the reality is we've spilled an awful lot of effort in this country over a considerable amount of time," the outgoing commander said.
+
+"The British public are very proud of the courage and sacrifice of the British armed forces here, but actually they should be just as proud of the achievements.
+
+"The reality is that we will be leaving Helmand a transformed province, indeed the whole of Afghanistan is transformed. We should be proud of that.
+
+"We've given the Afghans an opportunity. How they use that opportunity is down to them, but there's no doubt they've been given an opportunity."
+
+'Far superior'
+The Afghans had "taken the lead" over this summer's fighting and had sustained some casualties, but had done a "genuinely impressive job", he said.
+
+"There's no doubt the insurgents wanted to try and have a decisive effect this summer.
+
+"The fact that they have not been able to is down to the Afghan performance."
+
+He said Helmand province was still "a dangerous place", but said the casualties had not had a "decisive impact" on the police or army.
+
+His comments were echoed by some of the members of 2 Scots, who spent their tour of more than six months training and mentoring the Afghan National Police at Lashkar Gah.
+
+The local security forces were described by Warrant Officer William Garrick as "a lot better at what they do than we are".
+
+"They may not have the capability in terms of the vehicles, the manpower etc.
+
+"But in terms of common knowledge they are certainly far superior than we are."