Reversing: Secrets of Reverse Engineering / Edition 1

Reversing: Secrets of Reverse Engineering / Edition 1

by Eldad Eilam
ISBN-10:
0764574817
ISBN-13:
9780764574818
Pub. Date:
04/15/2005
Publisher:
Wiley
ISBN-10:
0764574817
ISBN-13:
9780764574818
Pub. Date:
04/15/2005
Publisher:
Wiley
Reversing: Secrets of Reverse Engineering / Edition 1

Reversing: Secrets of Reverse Engineering / Edition 1

by Eldad Eilam
$44.0 Current price is , Original price is $44.0. You
$44.00 
  • SHIP THIS ITEM
    Qualifies for Free Shipping
  • PICK UP IN STORE
    Check Availability at Nearby Stores
$14.30 
  • SHIP THIS ITEM

    Temporarily Out of Stock Online

    Please check back later for updated availability.

    • Condition: Good
    Note: Access code and/or supplemental material are not guaranteed to be included with used textbook.

Overview

Beginning with a basic primer on reverse engineering-including computer internals, operating systems, and assembly language-and then discussing the various
applications of reverse engineering, this book provides readers with practical, in-depth techniques for software reverse engineering. The book is broken into two parts, the first deals with security-related reverse engineering and the second explores the more practical aspects of reverse engineering. In addition, the author explains how to reverse engineer a third-party software library to improve interfacing and how to reverse engineer a competitor's software to build a better product.
* The first popular book to show how software reverse engineering can help defend against security threats, speed up development, and unlock the secrets of competitive products
* Helps developers plug security holes by demonstrating how hackers exploit reverse engineering techniques to crack copy-protection schemes and identify software targets for viruses and other malware
* Offers a primer on advanced reverse-engineering, delving into "disassembly"-code-level reverse engineering-and explaining how to decipher assembly language

Product Details

ISBN-13: 9780764574818
Publisher: Wiley
Publication date: 04/15/2005
Pages: 624
Product dimensions: 7.40(w) x 9.20(h) x 1.30(d)

About the Author

Eldad Eilam is a consultant in the field of reverse engineering. He assists clients with operating system and in-depth software reverse engineering, and has devoted several years to developing advanced reverse engineering techniques.

Read an Excerpt

Click to read or download

Table of Contents

Foreword vii

Acknowledgments xi

Introduction xxiii

Part I Reversing 101 1

Chapter 1 Foundations 3

What Is Reverse Engineering? 3

Software Reverse Engineering: Reversing 4

Reversing Applications 4

Security-Related Reversing 5

Malicious Software 5

Reversing Cryptographic Algorithms 6

Digital Rights Management 7

Auditing Program Binaries 7

Reversing in Software Development 8

Achieving Interoperability with Proprietary Software 8

Developing Competing Software 8

Evaluating Software Quality and Robustness 9

Low-Level Software 9

Assembly Language 10

Compilers 11

Virtual Machines and Bytecodes 12

Operating Systems 13

The Reversing Process 13

System-Level Reversing 14

Code-Level Reversing 14

The Tools 14

System-Monitoring Tools 15

Disassemblers 15

Debuggers 15

Decompilers 16

Is Reversing Legal? 17

Interoperability 17

Competition 18

Copyright Law 19

Trade Secrets and Patents 20

The Digital Millenium Copyright Act 20

DMCA Cases 22

License Agreement Considerations 23

Code Samples & Tools 23

Conclusion 23

Chapter 2 Low-Level Software 25

High-Level Perspectives 26

Program Structure 26

Modules 28

Common Code Constructs 28

Data Management 29

Variables 30

User-Defined Data Structures 30

Lists 31

Control Flow 32

High-Level Languages 33

C 34

C++ 35

Java 36

C# 36

Low-Level Perspectives 37

Low-Level Data Management 37

Registers 39

The Stack 40

Heaps 42

Executable Data Sections 43

Control Flow 43

Assembly Language 101 44

Registers 44

Flags 46

Instruction Format 47

Basic Instructions 48

Moving Data 49

Arithmetic 49

Comparing Operands 50

Conditional Branches 51

Function Calls 51

Examples 52

A Primer on Compilers and Compilation 53

Defining a Compiler 54

Compiler Architecture 55

Front End 55

Intermediate Representations 55

Optimizer 56

Back End 57

Listing Files 58

Specific Compilers 59

Execution Environments 60

Software Execution Environments (Virtual Machines) 60

Bytecodes 61

Interpreters 61

Just-in-Time Compilers 62

Reversing Strategies 62

Hardware Execution Environments in Modern Processors 63

Intel NetBurst 65

µops (Micro-Ops) 65

Pipelines 65

Branch Prediction 67

Conclusion 68

Chapter 3 Windows Fundamentals 69

Components and Basic Architecture 70

Brief History 70

Features 70

Supported Hardware 71

Memory Management 71

Virtual Memory and Paging 72

Paging 73

Page Faults 73

Working Sets 74

Kernel Memory and User Memory 74

The Kernel Memory Space 75

Section Objects 77

VAD Trees 78

User-Mode Allocations 78

Memory Management APIs 79

Objects and Handles 80

Named objects 81

Processes and Threads 83

Processes 84

Threads 84

Context Switching 85

Synchronization Objects 86

Process Initialization Sequence 87

Application Programming Interfaces 88

The Win32 API 88

The Native API 90

System Calling Mechanism 91

Executable Formats 93

Basic Concepts 93

Image Sections 95

Section Alignment 95

Dynamically Linked Libraries 96

Headers 97

Imports and Exports 99

Directories 99

Input and Output 103

The I/O System 103

The Win32 Subsystem 104

Object Management 105

Structured Exception Handling 105

Conclusion 107

Chapter 4 Reversing Tools 109

Different Reversing Approaches 110

Offline Code Analysis (Dead-Listing) 110

Live Code Analysis 110

Disassemblers 110

IDA Pro 112

ILDasm 115

Debuggers 116

User-Mode Debuggers 118

OllyDbg 118

User Debugging in WinDbg 119

IDA Pro 121

PEBrowse Professional Interactive 122

Kernel-Mode Debuggers 122

Kernel Debugging in WinDbg 123

Numega SoftICE 124

Kernel Debugging on Virtual Machines 127

Decompilers 129

System-Monitoring Tools 129

Patching Tools 131

Hex Workshop 131

Miscellaneous Reversing Tools 133

Executable-Dumping Tools 133

DUMPBIN 133

PEView 137

PEBrowse Professional 137

Conclusion 138

Part II Applied Reversing 139

Chapter 5 Beyond the Documentation 141

Reversing and Interoperability 142

Laying the Ground Rules 142

Locating Undocumented APIs 143

What Are We Looking For? 144

Case Study: The Generic Table API in NTDLL.DLL 145

RtlInitializeGenericTable 146

RtlNumberGenericTableElements 151

RtlIsGenericTableEmpty 152

RtlGetElementGenericTable 153

Setup and Initialization 155

Logic and Structure 159

Search Loop 1 161

Search Loop 2 163

Search Loop 3 164

Search Loop 4 165

Reconstructing the Source Code 165

RtlInsertElementGenericTable 168

RtlLocateNodeGenericTable 170

RtlRealInsertElementWorker 178

Splay Trees 187

RtlLookupElementGenericTable 188

RtlDeleteElementGenericTable 193

Putting the Pieces Together 194

Conclusion 196

Chapter 6 Deciphering File Formats 199

Cryptex 200

Using Cryptex 201

Reversing Cryptex 202

The Password Verification Process 207

Catching the “Bad Password” Message 207

The Password Transformation Algorithm 210

Hashing the Password 213

The Directory Layout 218

Analyzing the Directory Processing Code 218

Analyzing a File Entry 223

Dumping the Directory Layout 227

The File Extraction Process 228

Scanning the File List 234

Decrypting the File 235

The Floating-Point Sequence 236

The Decryption Loop 238

Verifying the Hash Value 239

The Big Picture 239

Digging Deeper 241

Conclusion 242

Chapter 7 Auditing Program Binaries 243

Defining the Problem 243

Vulnerabilities 245

Stack Overflows 245

A Simple Stack Vulnerability 247

Intrinsic Implementations 249

Stack Checking 250

Nonexecutable Memory 254

Heap Overflows 255

String Filters 256

Integer Overflows 256

Arithmetic Operations on User-Supplied Integers 258

Type Conversion Errors 260

Case-Study: The IIS Indexing Service Vulnerability 262

CVariableSet::AddExtensionControlBlock 263

DecodeURLEscapes 267

Conclusion 271

Chapter 8 Reversing Malware 273

Types of Malware 274

Viruses 274

Worms 274

Trojan Horses 275

Backdoors 276

Mobile Code 276

Adware/Spyware 276

Sticky Software 277

Future Malware 278

Information-Stealing Worms 278

BIOS/Firmware Malware 279

Uses of Malware 280

Malware Vulnerability 281

Polymorphism 282

Metamorphism 283

Establishing a Secure Environment 285

The Backdoor.Hacarmy.D 285

Unpacking the Executable 286

Initial Impressions 290

The Initial Installation 291

Initializing Communications 294

Connecting to the Server 296

Joining the Channel 298

Communicating with the Backdoor 299

Running SOCKS4 Servers 303

Clearing the Crime Scene 303

The Backdoor.Hacarmy.D: A Command Reference 304

Conclusion 306

Part III Cracking 307

Chapter 9 Piracy and Copy Protection 309

Copyrights in the New World 309

The Social Aspect 310

Software Piracy 310

Defining the Problem 311

Class Breaks 312

Requirements 313

The Theoretically Uncrackable Model 314

Types of Protection 314

Media-Based Protections 314

Serial Numbers 315

Challenge Response and Online Activations 315

Hardware-Based Protections 316

Software as a Service 317

Advanced Protection Concepts 318

Crypto-Processors 318

Digital Rights Management 319

DRM Models 320

The Windows Media Rights Manager 321

Secure Audio Path 321

Watermarking 321

Trusted Computing 322

Attacking Copy Protection Technologies 324

Conclusion 324

Chapter 10 Antireversing Techniques 327

Why Antireversing? 327

Basic Approaches to Antireversing 328

Eliminating Symbolic Information 329

Code Encryption 330

Active Antidebugger Techniques 331

Debugger Basics 331

The IsDebuggerPresent API 332

SystemKernelDebuggerInformation 333

Detecting SoftICE Using the Single-Step Interrupt 334

The Trap Flag 335

Code Checksums 335

Confusing Disassemblers 336

Linear Sweep Disassemblers 337

Recursive Traversal Disassemblers 338

Applications 343

Code Obfuscation 344

Control Flow Transformations 346

Opaque Predicates 346

Confusing Decompilers 348

Table Interpretation 348

Inlining and Outlining 353

Interleaving Code 354

Ordering Transformations 355

Data Transformations 355

Modifying Variable Encoding 355

Restructuring Arrays 356

Conclusion 356

Chapter 11 Breaking Protections 357

Patching 358

Keygenning 364

Ripping Key-Generation Algorithms 365

Advanced Cracking: Defender 370

Reversing Defender’s Initialization Routine 377

Analyzing the Decrypted Code 387

SoftICE’s Disappearance 396

Reversing the Secondary Thread 396

Defeating the “Killer” Thread 399

Loading KERNEL32.DLL 400

Reencrypting the Function 401

Back at the Entry Point 402

Parsing the Program Parameters 404

Processing the Username 406

Validating User Information 407

Unlocking the Code 409

Brute-Forcing Your Way through Defender 409

Protection Technologies in Defender 415

Localized Function-Level Encryption 415

Relatively Strong Cipher Block Chaining 415

Reencrypting 416

Obfuscated Application/Operating System Interface 416

Processor Time-Stamp Verification Thread 417

Runtime Generation of Decryption Keys 418

Interdependent Keys 418

User-Input-Based Decryption Keys 419

Heavy Inlining 419

Conclusion 419

Part IV Beyond Disassembly 421

Chapter 12 Reversing .NET 423

Ground Rules 424

.NET Basics 426

Managed Code 426

.NET Programming Languages 428

Common Type System (CTS) 428

Intermediate Language (IL) 429

The Evaluation Stack 430

Activation Records 430

IL Instructions 430

IL Code Samples 433

Counting Items 433

A Linked List Sample 436

Decompilers 443

Obfuscators 444

Renaming Symbols 444

Control Flow Obfuscation 444

Breaking Decompilation and Disassembly 444

Reversing Obfuscated Code 445

XenoCode Obfuscator 446

DotFuscator by Preemptive Solutions 448

Remotesoft Obfuscator and Linker 451

Remotesoft Protector 452

Precompiled Assemblies 453

Encrypted Assemblies 453

Conclusion 455

Chapter 13 Decompilation 457

Native Code Decompilation: An Unsolvable Problem? 457

Typical Decompiler Architecture 459

Intermediate Representations 459

Expressions and Expression Trees 461

Control Flow Graphs 462

The Front End 463

Semantic Analysis 463

Generating Control Flow Graphs 464

Code Analysis 466

Data-Flow Analysis 466

Single Static Assignment (SSA) 467

Data Propagation 468

Register Variable Identification 470

Data Type Propagation 471

Type Analysis 472

Primitive Data Types 472

Complex Data Types 473

Control Flow Analysis 475

Finding Library Functions 475

The Back End 476

Real-World IA-32 Decompilation 477

Conclusion 477

Appendix A Deciphering Code Structures 479

Appendix B Understanding Compiled Arithmetic 519

Appendix C Deciphering Program Data 537

Appendix D Citations 561

Index 567

From the B&N Reads Blog

Customer Reviews