The project has nine work-packages with eight public deliverables: D1.1, D2.1, D4.1, D5.1, D7.1, D7.2, D8.1, D9.1. A summary for each of the deliverables is given. For the full version please follow the corresponding download link.
Deliverable 1.1. Scenarios and detailed specification of M12 demonstration
The aim of this deliverable is to specify further the Month 12 demonstration of the CloPeMa project, with respect to the Description of Work. Much of the CloPeMa first year efforts will be spent on establishing the experimental platform (dual-arm robot system), develop methods and competencies at each partner site, and starting real cooperation between CloPeMa partners. The Month 12 demonstration should prove that CloPeMa partners reached the state-of-the-art level by mimicking capabilities demonstrated by others, i.e. pick a towel from a table and fold it. To bootstrap research activities, the tasks of the Month 12 demonstration scenario is broken down into relatively independent competencies to be provided by project partners. A part of this deliverable is devoted to the review of the related state-of-the-art, including literature in sensing, perception, manipulation and planning related to soft materials, such as textiles or garments. Several industrial application scenarios are presented as well. The report, finally, discusses the limitations of the state-of-the-art and highlight existing challenges that have to be confronted during the project.
The full deliverable can be viewed and downloaded at CloPeMa_D1.1_V1.1 as pdf.
Deliverable 1.2. Demonstrator Procedures and Results, Annotated Data Specification
This report describes the demonstration procedures of the CloPeMa project.
The demonstration procedures consist of two joint scenarios and a few
separate presentations. The two joint scenarios are motivated by laundry process.
We introduce sorting of garments before washing and folding of a garment. The report
contents list annotated datasets with short descriptions. The annotated datasets are
results of the project.
The full version of the deliverable is available at CloPeMa_D1.2_V0.1 in pdf.
Deliverable 2.1. Experimental testbed with dual robotic arm chain and its control
This report describes a prototype of an experimental testbed with dual robotic arm and the fundamental software necessary for the testbed use. Robot arm controllers are enhanced by a control program which provides mutual coordinated control and collision detection. Motion of the arms is planned automatically in accordance with a collision detection subsystem. The process of calibration and its utilisation is described too. The testbed systems have been developed by CVUT in close cooperation with NS. Proposed solutions are continuously being transferred to identical testbeds located at CERTH and UG.
The full version of the deliverable is classified confidential.
Deliverable 2.2. Experimental testbed with dual multifinger hands and its control using tactile feedback
This report describes CloPeMa experimental testbed allowing perception and dual arm manipulation with soft objects as garments. This report deals with force/torgue sensor, Year 2 Gripper with tactile sensing and rubbing capabilities. Having the force feedback enables to use off-the-shelf industrial arms as a force compliant robots. Related design, implementation and experiments are described in this report.
The full version of the deliverable is classified confidential.
Deliverable 2.3. Motion planning based on advanced sensors and reasononing
This report describes CloPeMa experimental testbed allowing
perception and dual arm manipulation. In particular, it deals with the new MotoROS
driver, improved planning algorithm and demonstration of the improvements. The
new driver increases the speed of the force feedback loop from 2 Hz to 10Hz.
The full version of the deliverable is classified confidential.
Deliverable 4.1. Binocular Robot Head Implementation and Operating Software
Deliverable 4.1 is an output of Task 4.1 in Workpackage 4, and comprises the design and implementation of the principal vision system for he Clopema clothing manipulation robot. This deliverable includes an investigation of potential camera and actuator configurations and the most appropriate configuration identified is used to implement a programmable binocular sensor head. In order to be suitable for quantitative range-finding within the workspace of the CloPeMa two-handed robot, this sensor head must be able to maintain dynamic calibration under active gaze control. The developed binocular vision head is supported by control software operating as nodes within the ROS environment to provide high-resolution stereo-pair image capture, lower resolution binocular streaming video, automatic camera convergence, range-map construction, camera calibration and SIFT feature extraction.
The full deliverable can be viewed and downloaded at CLoPeMa_4.1.pdf as pdf.
Deliverable 4.2. Photometric stereo gripper mounted sensor implementation with supporting documentation
This report describes the design and implementation of the photometric stereo sensor.
The sensor consists of affordable, off-the-shelf components and its main characteristic
is the very small size. It will be placed inside the gripper of the robot to infer the
material properties of fabrics by capturing their 3-D microstructure.
The full version of the deliverable is classified confidential.
Deliverable 5.1. Report on the Reasoning Mechanism and the Object Representation
This report describes techniques for robust extraction of persistent features such as folds, corners and edges from a noisy range image, followed by connecting these features into a topological structure that is mapped to a garment template by using knowledge and geometry. The resulting mapping allows detection of grasping affordances or planning robot movements that will reveal such affordances. Algorithms for garment shape recognition in a flat state are presented. The algorithms are designed to be robust to moderate deformations. Another algorithmic method recognizes a garment and registers its shape with a template even when the garment is folded on itself. The folding axis is also identified with respect to the template, thus allowing the planning of an unfolding sequence. First experiments exploiting only boundary shape primitives demonstrate promising results under significant occlusions of garment parts. Finally, a linguistic inspired architecture based on graph grammars is proposed to perform both bottom-up and top-down parsing of a garment image incorporating high-level knowledge of the garment structure.
The full deliverable can be viewed and downloaded at CloPeMa_D5.1_V1.0 as pdf.
Deliverable 5.2. Report and software for the reasoning mechanism in assotiation with learning and hardware control
This report describes Active Random Forests, a novel framework to address active vision
problems. State of the art focuses on best viewing parameters selection like viewpoint
or zooming based on single view classifiers. In contrast, we propose a multi-view
classifier where the process of taking actions for optimally changing viewing parameters
is inherent to the classification process. This has many advantages: a) the classifier
exploits the entire set of captured images and does not simply aggregate probabilistically
per view hypotheses; b) actions are based on learnt disambiguating image features
from all possible views and are optimally selected using the powerful voting scheme
of Random Forests and c) the classifier can take into account the costs of its actions.
The proposed framework is applied to the task of autonomously unfolding clothes by
a robot, addressing the problem of best viewpoint selection in classification, pose and
grasp point estimation of garments. We show great performance improvement compared
to random viewpoint selection and state of the art methods.
The full version of the deliverable is classified confidential.
Deliverable 6.1. Visual patterns
This report describes how CloPeMa explores visual information and uses it for the scene
understanding, planning and manipulation of a garment piece. First, visual sensors implemented at CloPeMa testbed are described. Second, the procedure for grasping a piece of garment is explicated. Third, buttons serving as features are utilized. Fourth, unfolding skills using random forests are described. Fifth, the garment description from a high resolution stereo head is reported. Sixth, a substantial improvement to the state of the art Berkeley garment model is suggested and its experimental validation documented.
The full version of the deliverable is available in pdf at CloPeMa_D6.1_V1.0.
Deliverable 6.2. Grasp patterns
This report describes CloPeMa skills involving grasping and basic manipulation of a garment. First, the new sensorized hand prototype and its capabilities for grasping and tactile exploration are demonstrated. Then the deliverable presents different vision-based techniques for finding suitable grasp points to facilitate high level planning and manipulation (e.g. folding, unfolding). Finally, low-level manipulation skills such as laying the garment on a table and flattening it are described. The performance is extensivelly evaluated on benchmark datasets.
The full version of the deliverable is available in pdf at CloPeMa_D6.2_V1.0.
Deliverable 7.1. Experimental testbed, basic functionality
Experimental testbed, basic functionality: This deliverable describes the basic hardwareon which all subsequent demonstrators will be built and which will be replicatedto the three sites that will develop the various components that will be incorporated toit. The testbed has been constructed on the CTU in Prague with a close cooperation ofthe NEO.
The full version of the deliverable is classified confidential.
Deliverable 7.2. First project demonstrator and its validation
This deliverable describes the first demonstrator of the CloPeMa project. Its purpose is to present the state of the project and to make a preliminary evaluation of the performance of the individual subsystems. The demonstrator shows the components of the system separately but a section is presented which demonstrates the first integration and interaction of at least some of the components.
The full deliverable can be viewed and downloaded at CloPeMa_D7.2_V0.7 as pdf.
Deliverable 7.3. Second project demonstrator and its validation
This deliverable describes the second demonstrator of the CloPeMa project. Its purpose
is to present the state of the project and to make a preliminary evaluation of the
system performance and function. The demonstrator is focused on a presentation of
the individual modules which have been integrated into one system and work side-byside.
The modules share the same hardware, data, computational results and services
to perform their tasks. Moreover, several modules which are supposed to be integrated
in the third year are presented separately.
The full deliverable can be viewed and downloaded in pdf in CloPeMa_D7.3_V1.0.
Deliverable 7.4. Third project demonstrator and its validation
This deliverable describes the third demonstrator of the CloPeMa project. Its purpose
is to present the state of the project and to make a preliminary evaluation of the system
performance and function. The demonstrator is focused on a presentation of the individual modules which have been integrated into one system and work side-by-side. The modules share the same hardware, data, computational results and services to perform their tasks.
The full deliverable can be viewed and downloaded in pdf in CloPeMa_D7.4_V1.0.
Deliverable 8.1. CloPeMa Website Report
CloPeMa’s project website went online on May 1st, 2012 and can be found under www.clopema.eu. The website features general public information about the project as well as documentation for the developed tools and multimedia material. The website uses common web-application service, such as SourceForge’s code repository, WordPress’s news blog and MediaWiki documentation to presents its contents to the visitors.
The full deliverable can be viewed and downloaded at CloPeMa_D8.1_V1.0 as pdf.
Deliverable 9.1. Quality Management Report
This document constitutes the prime reference point for all management and quality assurance procedures of the CloPeMa project. The document includes a project summary section, which provides information on the main project facts, namely the project’s work breakdown, its interdependencies, and the project timetable regarding deliverables and milestones, along with responsible partners. After this, the Project Management Plan is described. This provides a concrete view of all foreseen project management procedures to be adopted by the project coordinator and the beneficiaries. The remaining part is dedicated to the description of the Quality Plan, which covers the foreseen procedures for assessing the progress of the work within the project, along with corrective actions and contingency planning in case of deviations. It is strongly emphasized that this is an ongoing document that is being evolved along with the project progress and will be regularly updated to reflect up-to-date information.
The full deliverable can be viewed and downloaded at CloPeMa_D9.1_V0.9 as pdf.